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Foreword 


The  F/A-22  Raptor  is  a  new  weapons  system  replacing 
the  F-15C  Eagle.  Its  operational  debut  in  2005  comes  at  a 
time  of  constrained  budgets,  a  changing  global  threat  envi¬ 
ronment,  and  the  ongoing  global  war  on  terrorism  (GWOT). 
Because  of  the  current  security  environment,  the  aircraft’s 
more  than  1 5  years  of  development,  and  the  close  scrutiny 
of  the  F /A-22’s  test  and  evaluation  (T&E)  program,  the  US 
Air  Force  needs  the  Raptor’s  initial  operational  capability 
(IOC)  status  to  be  successful.  One  means  of  achieving  this 
is  to  recognize  and  implement  the  lessons  learned  from 
America’s  current  air  superiority  fighter,  the  F-15  Eagle. 
And  just  perhaps  the  overall  effect  might  be  to  challenge 
the  US  Air  Force’s  approach  to  major  weapons-system 
development. 

The  F/A-22’s  development,  testing,  and  IOC  declaration 
at  Langley  AFB,  Virginia,  in  December  2005  closely  parallels 
the  F-15A’s  experience  of  29  years  ago.  This  paper  provides 
background  information  on  both  aircraft,  their  T&E 
processes,  and  their  first  operational  assignments  to 
Langley  AFB.  Comparisons  are  made,  differences  high¬ 
lighted,  and  recommendations  offered.  While  it  may  appear 
that  everything  about  the  F/A-22  is  new,  the  path  to  its 
IOC  is  well  worn. 

A  fresh  look  at  F-15A  development  yields  valuable 
insights  for  emerging  twenty-first-centuiy  USAF  weapons 
systems.  Surprisingly,  the  overall  lessons  of  the  F-15A  in 
1976  still  apply  to  the  F/A-22  of  2005. 

The  1st  Fighter  Wing  is  again  tasked  with  birthing  a 
major  weapons  system,  and  the  difficulties  of  29  years  ago 
will  again  be  met  and  overcome  with  the  proven  successful 
combination  of  leadership  and  teamwork.  However,  this 
paper  not  only  focuses  on  the  1st  Fighter  Wing’s  path  to 
IOC  but  also  considers  the  USAF’s  T&E  process. 

Comparison  of  the  two  T&E  processes  has  identified 
that  the  F-15A  and  F/A-22  used  the  same  model  for  T&E 
(developmental  testing  and  then  operational  testing).  The 
programs  faced  milestones  that  were  used  to  keep  the  pro¬ 
grams  on  track  and  aligned  with  the  contracts.  Both  had 
to  react  to  changing  oversight  direction,  both  needed 
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greater  operational  involvement  in  the  early  phases  of  test¬ 
ing,  and  both  completed  their  initial  T&E  processes  imme¬ 
diately  before  IOC  at  Langley  AFB,  with  further  T&E  to  follow. 
Both  programs  introduced  significant  changes  to  the  T&E 
process,  but  striking  similarities  after  nearly  three  decades 
could  point  to  a  need  for  major  transformational  change  in 
the  USAF’s  T&E  process. 

In  addition,  the  author  identifies  specific  recommenda¬ 
tions  that  could  improve  the  IOC  of  new  weapons  systems. 
Some  are  relatively  intuitive:  units  declaring  IOC  should 
gather  data  from  other  units  undergoing  the  same  process, 
capture  the  experience  in  the  form  of  histories,  and  make 
an  effort  to  get  the  aircraft  “on  the  road”  to  better  introduce 
it  to  the  Combat  Air  Forces  (CAF).  A  suggestion  to  imple¬ 
ment  an  IOC  advisory  board  to  keep  senior  leaders 
informed  of  actions  and  benefit  from  senior  officer  experi¬ 
ence  also  may  have  merit. 

Recommendations  such  as  improving  the  handoff  between 
testers  and  operators  or  redelining  the  meaning  and  process 
of  IOC  are  far-reaching  suggestions,  and  their  implemen¬ 
tation  would  have  an  impact  on  organizations,  processes, 
and  funding  that  could  transform  the  way  the  USAF 
acquires  new  weapons  systems.  Finally,  Colonel  Mott  is 
clearly  on  target  when  he  concludes  that  the  initial  Raptor 
units,  whether  test  or  operational,  must  capture,  challenge, 
and  doctrinally  address  the  operational  mind-set  for  employ¬ 
ing  the  F/A-22.  F/A-22s  should  not  simply  use  F-15  tac¬ 
tics;  the  F/A-22’s  transformational  nature  must  be  nur¬ 
tured  and  exploited  to  provide  maximum  combat  effect  for 
the  war  fighter. 

Surprisingly,  29  years  have  not  completely  changed  the 
nature  of  bringing  a  new  weapons  system  to  IOC  at 
Langley  AFB.  The  repetition  of  similar  challenges  such  as 
T&E  timeliness  and  completion,  reaction  to  changes 
directed  by  oversight,  assumptions  about  sortie  production 
rates,  maintenance  factors,  and  technology  choke  points  all 
combine  to  call  for  transformation  of  the  USAF  path  to 
weapons-systems  IOC.  This  study  calls  for  transformational 
change  to  be  considered  within  either  the  T&E  process  or 
the  buildup  to  IOC  such  that  IOC  declarations  of  future 
weapons  systems  might  be  improved.  By  doing  so,  future  war 
fighters  will  get  a  better  weapons  system  at  IOC  than  they 


have  either  in  the  past  or  currently.  This  study  should 
encourage  a  critical  look  at  the  F/A-22’s  road  to  IOC, 
prompting  thinking  on  changes  that  might  be  made  to 
other  new  weapons  systems  like  the  Joint  Strike  Fighter  or 
CV-22. 

As  with  all  Maxwell  Papers,  this  research  is  provided  in 
the  spirit  of  academic  freedom,  open  debate,  and  serious 
consideration  of  the  issues.  We  encourage  your  responses. 


Major  General,  USAF 
Commandant,  Air  War  College 
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F-15A  versus  F/A-22  Initial 
Operational  Capability 

A  Case  for  Transformation 


You  ask  me  a  question  about .  .  .  air-to-air  tactics.  All  I  can  say 
is  that  the  cardinal  rules  of  air  tactics  haven’t  changed  one  bit 
since  World  War  I  ..  .  formation  integrity,  good  eyesight,  ag¬ 
gressiveness,  training,  of  course,  a  plan  of  action  helps,  in¬ 
tegrity  of  flight.  .  .  .  All  these  things  are  well  known  to  fighter 
pilots.  Unfortunately,  they  are  not  so  well  understood  by  other 
members  of  the  Air  Force  or  by  people  who  sometimes  design 
airplanes  and  buy  them. 

— Col  Robin  Olds,  Commander 
8th  Tactical  Fighter  Wing 
12  July  1967 

Although  air-combat  tactics  have  not  changed  during 
the  last  100  years  of  manned  flight,  aircraft  certainly  have. 
The  purpose  of  air-superiority  fighters  remains  the  same — 
to  maximize  the  warrior’s  ability  to  kill  the  enemy  in  the  air 
and  achieve  the  critical  war-fighting  enabler  of  air  domi¬ 
nance.  However,  aerospace  design,  testing,  and  engineering 
development  have  combined  to  make  the  modern  jet  fighter 
a  more  complex  and  costly  machine  than  ever  before. 

In  2005  we  see  the  dawn  of  a  new  fighter — one  that  is 
specifically  designed  to  dominate  in  every  air  battle,  to  pro¬ 
vide  the  United  States  an  undeniable  ability  to  operate  over 
enemy  territory,  to  exploit  an  enemy’s  airspace,  and  to 
hold  his  entire  nation  at  risk.  That  aircraft  is  the  Lockheed 
F/A-22  Raptor.  At  a  program  cost  of  $72  billion  in  fiscal 
year  2004  dollars,  an  F/A-22  costs  twice  what  an  F-15C 
costs  in  constant  dollars  during  the  later  stages  of  its  pro¬ 
duction  and  at  a  production  rate  half  that  of  the  F-15.1 

The  F/A-22  is  a  multibillion-dollar  acquisition  program 
that  is  the  center  of  controversy  ranging  from  within  the 
Department  of  Defense  (DOD)  to  the  United  States  House 
and  Senate.  The  F/A-22  has  the  technology  to  dominate  in 
air  combat  but  at  what  cost?  Is  the  aircraft  an  overpriced 
Cold  War  relic?  Is  it  worth  the  investment  in  light  of  ever- 
increasing  costs  associated  with  the  current  global  war  on 
terrorism  (GWOT)?  Even  as  late  as  December  2004  the 
total  buy  is  uncertain,  according  to  a  defense  consultant: 
“If  the  Pentagon’s  reported  plan  to  reduce  F/A-22  orders  to 
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F/A-22  Raptor 


USAF  Photo 


about  160  planes  is  approved,  production  cuts  would  start 
taking  effect  after  the  2007  budget  year  and  production 
could  end  by  20 10.”2  The  military  and  political  debate  sur¬ 
rounding  the  F/A-22  has  produced  tremendous  interest  in 
its  acquisition,  oversight,  and  testing  programs.  Yet,  how 
different  will  the  first  operational  squadron  of  F/A-22s  in 
2005  be  from  the  last  grand  opening  of  a  squadron  of  air- 
superiority  fighters  at  Langley  AFB,  Virginia? 

In  1976  the  1st  Tactical  Fighter  Wing  (TFW)  declared  ini¬ 
tial  operational  capability  (IOC)  in  the  27th  Tactical  Fighter 
Squadron  (TFS)  with  the  McDonnell  Douglas  F-15A  Eagle. 
Like  the  F/A-22,  the  F-15A  was — and  still  is — an  out¬ 
standing  air-superiority  fighter.  It  was  a  generational  leap 
in  technology  (beyond  the  F-4  Phantom  it  replaced)  and  the 
center  of  a  new,  if  not  controversial,  acquisition  process.  The 
similarities  are  too  great  to  evade  the  question:  has  the 
USAF  learned  anything  from  history?  Or,  are  the  F/A-22 
pilots  of  2005/2006  going  to  relearn  the  same  lessons  of 
1976? 

A  complete  comparison  of  the  F-15A’s  and  the  F/A-22’s 
acquisition  programs  is  beyond  the  scope  of  this  paper. 
This  paper’s  focus  is  first  on  the  background  events  that 
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F-15A  Eagle  USAF  Photo 


led  to  the  development  of  the  F-15A  air-superiority  fighter. 
It  then  considers  the  F-15A’s  test  and  evaluation  (T&E) 
process  and  buildup  to  IOC  at  Langley  AFB  in  1976  and 
repeats  this  evaluation  for  the  F/A-22.  Finally,  it  recom¬ 
mends  adjustments  to  the  F/A-22’s  path  to  full  opera¬ 
tional  capability  (FOC)  and  for  the  Joint  Strike  Fighter’s 
(JSF)  IOC. 

The  declaration  of  IOC  for  both  aircraft  heralded  a  new 
era  in  American  fighter  technology  and  capability  and  is  a 
critical  point  for  review.  By  distilling  the  development  process 
of  the  F- 1 5A  to  its  culmination  as  a  weapons  system  ready 
for  combat  and  comparing  it  to  that  of  the  F/A-22,  it  is 
possible  to  assess  the  F/A-22’s  efficacy  at  IOC.  Obviously 
weapons  systems  introduced  in  1976  have  matured  with 
time  in  the  field,  and  the  same  is  true  in  2005  with  the 
F/A-22.  But  should  not  this  information  age  we  live  in  pro¬ 
duce  a  more  complete,  more  capable  system  at  IOC  than 
we  had  29  years  ago? 

F-15A  Eagle  Acquisition  Process 

Procurement  of  the  F-15A  weapons  system  was  a  dra¬ 
matic  change  for  the  USAF.  Behind  lay  the  tragedy  of  the 
Vietnam  War,  the  abortive  attempt  in  the  F-lll  to  build 
one  fighter  for  both  the  US  Navy  and  the  USAF,  and  the 
costly  (and  controversial)  C-5A  Galaxy  acquisition  process.3 
Ahead  lay  the  F-16  low-cost,  multirole  fighter;  the  A- 10 
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tank-busting  attack  aircraft;  and  the  on-again,  off-again 
B- 1  strategic  bomber.  The  lessons  learned  with  the  F- 1 5A 
would  serve  as  a  guidepost  as  the  Cold  War  matured  in  the 
1980s.  The  F-15A’s  format  for  development,  engineering, 
and  even  testing  would  leave  its  mark  on  the  future  F /A-22 
program. 

The  first  step  to  understanding  the  F-15A’s  acquisition 
process  is  to  examine  the  decision  to  build  a  fighter  with  a 
single  mission.  What  drove  the  USAF  to  write  a  require¬ 
ment  for  and  Congress  to  fund  the  development  of  an  air- 
superiority  fighter?  The  answer  is  found  in  the  history  of 
the  USAF  and  its  combat  experiences  of  World  War  II, 
Korea,  and  Vietnam.  During  World  War  II,  the  Army  Air 
Corps  started  with  a  doctrine  of  ground  attack:  “United 
States  War  Department  Field  Manual  31-35,  Aviation  in 
Support  of  Ground  Forces,’  9  April  1942,  allocated  aircraft 
resources  to  ground  units  and  placed  pursuit  aviation 
under  Army  control.  It  specifically  prescribed  that  air- 
power’s  mission  was  to  attack  ‘the  most  serious  threat  to 
the  operations  of  the  supported  ground  force.’”4 

This  apportionment  of  fighters  to  ground  commanders 
contributed  to  defeat  at  Kasserine  Pass  and  resulted  in  a 
change  in  doctrinal  organization  in  January  1943  that  was 
modeled  after  the  Royal  Air  Force.5  Suddenly,  new  doctrine 
ranked  “the  gaining  of  air  superiority  .  .  .  the  first  require¬ 
ment  for  the  success  of  any  major  land  operation,  placing 
this  function  ahead  of  tactical  air’s  interdiction  and  close 
air  support  missions.”6  From  North  Africa,  the  role  of  the 
fighter  continued  to  be  focused  on  air  superiority  by  sup¬ 
porting  the  USAF’s  daylight  precision-bombing  campaign 
against  Germany.  The  influence  of  the  P-47  Thunderbolt 
and  P-51  Mustang  on  the  air  war  in  Europe  is  well  known 
and  ultimately  led  to  the  establishment  of  air  superiority 
and  the  declaration  of  the  supreme  commander,  Gen 
Dwight  D.  Eisenhower,  that  “if  you  see  fighter  aircraft  over 
you,  they  will  be  ours.”7 

The  importance  of  air  superiority  continued  in  the  Korean 
War  as  the  advent  of  the  MiG- 1 5  upped  the  ante  for  tech¬ 
nological  supremacy  over  the  Korean  peninsula.  A  harbinger 
of  air  superiority’s  importance  came  when  B-29  attacks 
were  operationally  restricted,  even  to  the  point  of  flying 
only  at  night  due  to  their  being  shot  down  by  MiG- 15s.8 
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The  air  war  over  “MiG  Alley”  and  the  Yalu  River  produced 
fierce  fighting  between  the  F-86  Sabre  and  the  MiG- 15. 
The  USAF’s  F-86  Sabre  proved  equal  to  the  task,  ulti¬ 
mately  achieving  a  kill  ratio  of  10  to  one.9 

The  war  in  Vietnam  saw  the  critical  move  away  from  air- 
superiority  fighters  like  the  F-86.  The  1960s  saw  a  rise  of 
the  “bomber  generals”  that  focused  Tactical  Air  Command 
(TAC)  on  nuclear  delivery  and  a  gradual  dulling  of  the 
skills  of  fighter  air  combat,  commonly  known  as  “dogfight¬ 
ing.”  Hanson  Baldwin  writes  in  the  introduction  to  Thud 
Ridge  that  “there  had  been  too  much  dependence  on  nu¬ 
clear  weapons  and  ‘the  bomber  generals’  in  the  Air  Force 
had  long  down-graded  the  tactical  air  arm.”10  Worse,  the 
procurement  of  aircraft  resulted  in  weapons  systems  ill 
suited  for  the  conventional  war  over  North  Vietnam.  Col 
Walter  Boyne,  USAF,  retired,  observes  that  “out  of  the  833 
production  aircraft,  some  350  F-105’s  were  lost  to  combat 
or  other  operational  causes.”11  Obviously,  all  F-105  com¬ 
bat  losses  cannot  be  attributed  directly  to  its  design  and 
suitability  for  combat  over  North  Vietnam,  but  the  sheer 
number  of  F-105  losses  is  indicative  of  problems  with  the 
aircraft’s  operational  record. 

While  the  F-4  Phantom  did  achieve  limited  air  superiority 
over  North  Vietnam,  it  was  still  not  perfectly  designed  for 
air  combat  in  that  it  was  missing  a  gun,  a  fact  that  was  not 
lost  on  Colonel  Olds  in  the  skies  over  Hanoi:  “I  gnash  my 
teeth  in  rage  to  think  how  much  better  this  wing  could 
have  done  had  we  acquired  a  gun-carrying  capability  ear¬ 
lier.”12  Fighter  pilots  in  Vietnam  found  themselves  in  a 
shooting  war  with  a  motivated  foe,  and  they  didn’t  have  the 
right  aircraft  for  the  mission.  That  dissatisfaction  with  the 
air  war  in  Vietnam  led  to  a  great  many  changes  in  the 
USAF  in  the  1970s  and  1980s.  The  desire  to  have  a  supe¬ 
rior  dogfighting  aircraft  led  to  the  design  of  the  F- 1 5A: 

Although  air  superiority  remained  the  “prerequisite”  for  conducting 
any  air  operation.  General  Bruce  K.  Holloway,  the  Air  Force  Vice 
Chief,  wrote  in  1968  that  plans  to  develop  a  new  day  fighter  were 
continually  sacrificed  in  favor  of  interceptor  and  fighter-bomber  de¬ 
signs:  “Penetration  was  more  important  than  maneuverability,  ord¬ 
nance  load-carrying  capability  more  important  than  armament,  alert 
status  more  important  than  sustained  sortie  rates.  The  tactical 
fighter  became  less  and  less  an  air  superiority  system  and  more 
and  more  what  was  once  called  an  attack  aircraft.”13 
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By  the  late  1960s,  the  stage  was  set  for  the  resurgence 
of  single-mission  aircraft,  notably  the  F-15  and  the  A- 10. 
While  the  debate  between  “not  a  pound  for  air  to  ground” 
versus  a  multirole  fighter  continues  today,  the  USAF  pur¬ 
sued  a  single-mission  air -superiority  fighter  with  the  F-15A 
and  again  with  the  F-22.  This  was  to  last  until  2002  when 
the  F-22  Raptor  was  redesignated  the  F/A-22  by  the  chief 
of  staff  of  the  Air  Force  (CSAF).14 

Armed  with  an  understanding  of  why  the  F-15A  was 
built,  it  is  appropriate  to  look  at  how  it  was  built.  The  birth 
of  the  F-15A  came  at  a  time  when  there  was  great  concern 
for  the  ability  of  the  government  to  successfully  run  a  large 
aircraft-acquisition  program.  According  to  Lt  Col  Edgar  M. 
Lewis,  USAF,  retired,  “The  Air  Force  was  being  severely 
criticized  in  the  Research  and  Development/Program  Man¬ 
agement  area  because  of  the  F-l  1 1  and  C-5  problems.  The 
F-lll  cost  over-runs,  schedule  delays,  technical  difficulties 
and  operational  introduction  problems  have  been  well 
documented  in  various  Air  Force  and  TAC  .  .  .  documents. 
The  cost  and  schedule  problems  with  the  C-5  and  in  par¬ 
ticular  the  wing  fatigue  problem  were  in  the  headlines  in 
1969. ”15  In  addition,  the  F-15  was  the  technological  lead 
agent  for  a  family  of  systems  developed  during  the  later 
decades  of  the  Cold  War.  In  1973  Col  Thomas  G.  Mclnerney 
noted  that  “the  F-15  engine,  radar,  avionics  (tactical  elec¬ 
tronic  warfare  system,  internal  navigation  system),  and 
gun  development  programs  will  provide  a  family  of  systems 
that  will  be  available  for  many  other  defense  requirements 
throughout  the  1970’s  and  early  1980’s,  but  for  which  the 
F-15  program  has  absorbed  the  brunt  of  development 
costs.”16 

Concerns  for  the  F- 1 5A  program  and  the  technical  chal¬ 
lenges  faced  by  the  engineers  can  be  grouped  into  three 
categories:  “blueline”  (or  streamlined)  management,  sys¬ 
tem  program  office  (SPO)  and  TAC  relationships,  and  cost 
versus  contract.  Analysis  within  these  areas  not  only  sheds 
light  on  the  F-15A’s  history  but  also  provides  parallels  to 
the  F/A-22.  Prior  to  1969,  DOD  systems-acquisition  policy 
consisted  of  highly  centralized  management  and  total- 
package  procurement. 17  The  change  in  policy  implemented 
by  the  DOD’s  senior  leaders  resulted  in  decentralized  au¬ 
thority  for  programs  and  a  use  of  prototyping  rather  than 
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paper  studies  to  reduce  risk.18  This  command  relationship, 
when  coupled  with  blueline  management,  meant  that  pro¬ 
gram  monitors  could  rapidly  gain  access  to  senior  USAF 
decision  makers:  “The  F-15  was  put  under  the  ‘Streamlined 
management’  approach,  wherein  the  System  Program  Di¬ 
rector  (SPD)  reported  directly  to  the  Commander,  Air  Force 
Systems  Command  [AFSC],  who  in  turn,  reported  directly 
to  the  Air  Force  Chief  of  Staff  and  Secretary  and  the  As¬ 
sistant  Secretary  of  Defense  (Mr.  Packard).”19 

Next  the  F-15A  SPO  (the  developer)  and  TAC  (the  cus¬ 
tomer/user)  established  formal  relationships  that  allowed 
TAC  visibility  into  the  development  process.  The  efficacy  of 
these  relationships  remains  unclear.  Research  indicates  a 
variety  of  comments.  According  to  Colonel  Mclnerney,  in 
1973  “the  TAC/F-15  SPO  relationship  .  .  .  [was]  good,”  which 
he  concluded  from  an  interview  with  Brig  Gen  Walter 
Paluch,  deputy  chief  of  staff  for  requirements,  Headquarters 
TAC.20  Yet,  by  1975  Headquarters  TAC  had  initiated 
monthly  meetings/briefings  to  improve  F-15  SPO /TAC 
communication.21  That  same  year  Gen  David  C.  Jones,  the 
chief  of  staff,  became  involved  in  the  detailed  process  of  es¬ 
tablishing  “Super-PARs,”  or  program  assessment  reviews, 
to  assess  major  weapons  systems.22 

Further,  the  History  of  the  F-15  Eagle  report  that  Head¬ 
quarters  TAC  generated  addressed  organizational  hin¬ 
drances  to  full  coordination  between  TAC  and  the  F-15 
SPO.  At  Wright  Patterson  AFB,  Ohio,  “the  senior  managers/ 
directors  were  products  of  the  Green  Door/Blackbird 
(secret/closed  access)  society.  The  natural  tendency  was  to 
maintain  the  same  closed  society,  i.e.:  don’t  discuss 
progress/problems  with  anyone  outside  of  a  very  close  knit 
group.”23  This  philosophy  prevailed  at  Headquarters  TAC 
as  well: 

The  secretiveness  even  pervaded  the  TAC  Staff.  Problems  were 
known  and  discussed  only  at  the  Director/DCS  and  Command  sec¬ 
tion  levels.  The  responsible  Staff  Officer  often  found  out  about  a 
problem  by  accident,  deduction  or  after  it  was  resolved.  The  closed 
loop  was  insisted  on  by  the  F-15  SPO,  or  the  flow  of  information 
would  stop.  In  this  manner  “bad  press”  was  avoided,  but  conversely 
“good  press”  was  not  available  to  engender  support  for  the  F-15 
program  throughout  the  Air  Force.24 
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Clearly,  coordination  was  an  issue  in  1975-76.  If  this 
problem  continues  today,  consequences  could  be  signifi¬ 
cant.  Aircraft  are  not  developed  in  a  short  period  of  time: 
technology,  management,  congressional  oversight,  leader¬ 
ship,  and  even  roles  and  missions  can  change  from  the 
time  that  the  Operational  Requirements  Document  (ORD) 
is  signed  until  the  aircraft’s  first  flight.  There  should  be  a 
means  to  balance  program  changes  versus  engineering 
commitments  during  the  development  process.  Without 
proper  controls,  costs  can  increase;  even  worse,  the  com¬ 
mitment  to  purchase  the  entire  airframe  can  erode.  Wit¬ 
ness  the  2004  US  Army  decision  to  cancel  the  Comanche 
attack  helicopter  and  then  redirect  the  funds  to  other  as¬ 
pects  of  US  Army  aviation  requirements.25  The  F /A-22  has 
also  been  crippled  by  cost  overruns — “cost  growth  has 
been  a  problem  for  the  Raptor  program.  Most  cost  in¬ 
creases  have  been  due  to  government  decisions  that 
changed  requirements,  delayed  development,  reduced  pro¬ 
duction  rates  or  cut  the  size  of  the  planned  fleet.  Costs  also 
have  been  increased  by  technological  challenges  that  were 
not  fully  anticipated.”26 

After  streamlined  management  and  TAC-SPO  relations, 
another  important  factor  in  ensuring  the  F-15A’s  success 
was  in  its  funding  and  contract  development.  The  F-15A 
program's  contract  is  full  of  buzzwords  that  today  are  fa¬ 
miliar  to  acquisition  officers.  At  the  time,  they  were  inno¬ 
vative  and  a  result  of  lessons  learned  from  the  F-l  1  l’s  and 
the  C-5’s  program  shortcomings.  The  F-15  would  be  a  cost 
plus  incentive  fee  (CPIF)  for  development,  and  critical  sub¬ 
systems  were  developed  competitively  and  selected  on  the 
basis  of  performance  in  a  “fly-off.”27  Milestones — or  measures 
of  performance — were  included  at  significant  development 
points  to  allow  for  decisions  before  further  development.  A 
Cost/Schedule  Control  System  (C/SCS)  was  implemented 
that  “provided  meaningful  performance  measurement  sta¬ 
tus  by  comparing  an  established  budget  baseline  plan  with 
actual  resources  expended.”28  The  F-15  program  was  not 
free  from  difficulties  as  its  F-l 00  engine  had  numerous  de¬ 
velopmental  problems  early  in  its  history,  yet  Mclnerney 
says  in  his  1973  research  that  “this  is  probably  one  of  the 
best  contracts  ever  written  on  such  a  complex  development 
program  despite  the  cost  growth  on  the  engine.”29  It  would 
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not  be  an  understatement  to  say  that  the  lessons  learned 
from  the  F-15’s  acquisition  form  a  foundation  that  endures 
today.  As  discussed  in  the  next  section,  perhaps  the  biggest 
aspect  of  the  F-15A’s  development  program  that  affected 
the  IOC  was  the  requirement  to  “fly  before  you  buy.” 

F-15A  Test  and  Evaluation  Process 

Fly  before  you  buy  acquisition  means  to  develop  pro¬ 
gram  milestones  that  demonstrate  an  appropriate  level  of 
technical  and  flight-test  confidence  before  decisions  are 
made  to  further  obligate  US  government  funds.30  Not  only 
does  this  develop  a  flight-test  program  that  supports  the 
aircraft’s  engineering  and  production,  it  also  develops  op¬ 
erational  insight  before  the  weapons  system  reaches  IOC. 
The  different  T&E  steps  occurred  at  different  bases,  were 
flown  by  different  pilots,  overlapped  in  content  and  scope, 
and  yielded  a  four -year  process  from  first  flight  to  IOC  at 
Langley  AFB  (fig.  1). 

Contractor  Developmental  Test  and  Evaluation 

Contractor  developmental  test  and  evaluation  (CDT&E) 
was  conducted  from  first  flight  on  27  July  1972  to  the  end 
of  1974. 31  This  piece  of  the  overall  development  was  con¬ 
ducted  by  contractor  pilots  (McDonnell  Douglas  Corpora¬ 
tion)  at  Edwards  AFB  in  California.  As  would  be  expected, 
this  phase  of  testing  focused  on  translating  the  F-15A’s  de¬ 
signed  performance  into  actual  aircraft-handling  qualities. 
While  not  the  focus  of  this  paper,  some  of  the  findings  dur¬ 
ing  this  period  resulted  in  changes  (called  engineering 
change  proposals  [ECP])  to  the  raked  wingtips,  the  dog¬ 
tooth  horizontal  stabilators,  the  enlarged  speed  brake,  and 
20  other  engineering  or  manufacturing  refinements.32 

Air  Force  Developmental  Test  and  Evaluation 

Similar  to  the  contractor  tests,  Air  Force  DT&E  (AFDT&E) 
ran  from  April  1974  to  mid- 1975. 33  Conducted  by  USAF  pi¬ 
lots  at  Edwards  AFB,  it  again  focused  on  the  airframe  and 
the  basics  of  getting  the  components  to  work  together  as 
prescribed  by  the  engineers.  In  this  phase  of  development, 
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First  Flight 

TAC  1a  Eagle  1b 
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74  1 75  76'i  77  78 

Contractor  Developmental  Test 

1 - 1 

and  Evaluation  (CDT&E) 

Air  Force  Developmental  Test 

1  ~l 

and  Evaluation  (AFDT&E) 

Initial  Operational  Test 

□ 

and  Evaluation  (IOT&E) 

Follow-on  Operational  Test 

1  1 

and  Evaluation  (FOT&E) 

Air  Intercept  Missile  Evaluation/ 

□ 

Air  Combat  Evaluation  (AIMVAL/ACEVAL) 

Langley  Initial  Operational  Capability  (IOC)  1 _ 1 

Ready  Eagle  c 

i - 1 

Langley  Operational  Readiness 

nspection  (ORI)  □ 

a  TAC  1  was  the  first  F-1 5  to  be  delivered  to  Luke  AFB. 

b  Eagle  1  was  the  first  F-15  delivered  to  Langley  AFB. 

c  Ready  Eagle  prepared  pilots  and  jets  to  go  to  Bitburg  AB,  Germany. 

Figure  1.  F-15A  developmental  test  (DT)/operational  test  and 
evaluation  (OT&E)  schedule  and  milestones.  (Data  compiled 
from  sources  cited  in  notes  3,  4,  33,  49,  and  53.) 


the  F-15  SPO  took  a  step  back  and  allowed  the  Air  Force 
Test  and  Evaluation  Center  (AFTEC)  to  take  the  lead  in 
testing  the  F-15A.  Contractor  and  Air  Force  DT&E  pro¬ 
duced  a  common  goal  but  with  a  different  focus.  The  con¬ 
tractor  focused  on  meeting  the  USAF’s  requirements  while 
at  the  same  time  refining  and  developing  production  meth¬ 
ods  at  the  factory.  Although  the  USAF  was  also  concerned 
with  qualifying  the  design  versus  contractual  require¬ 
ments,  it  was  more  concerned  with  operating  the  aircraft 
with  minimum  contractor  support.34 

A  key  point  to  F-15A  testing  is  that  CDT&E  and 
AFDT&E  occurred  hand-in-hand.  They  both  flew  at  Ed¬ 
wards  AFB,  and  both  had  a  clear  arrangement  for  test  ob¬ 
jectives  and  processes.  But  on  11  February  1971,  the 
deputy  secretary  of  defense  established  a  new  test  policy 
for  the  Air  Force  that  required  testing  by  the  operator,  in 
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addition  to  that  performed  by  the  contractor  and  developer 
of  the  airframe.35 

Initial  Operational  Test  and  Evaluation 

Initial  operational  T&E  (IOT&E)  was  a  critical  and  yet 
unique  phase  in  the  development  of  the  F-15A.  It  was  a 
vital  complement  to  contractor  and  Air  Force  testing  that 
let  the  operator — TAC — have  a  say  in  the  evaluation  process. 
A  1977  AFSC  history  explains  that  the  “F-15  IOT&E  was 
jointly  conducted  with  the  Air  Force/contractor  develop¬ 
ment,  test,  and  evaluation.  This  IOT&E  was  Air  Force  di¬ 
rected,  TAC  conducted,  and  AFTEC  monitored.  The  IOT&E 
provided  estimates  of  system  operational  effectiveness  and 
suitability  and  also  identified  the  need  for  modifications 
early  in  the  acquisition  process.”36 

Now  this  description  of  IOT&E  sounds  great,  but  a  ‘‘peel¬ 
ing  of  the  onion  skin”  reveals  a  few  unmistakable  factors. 
A  briefing  by  Lieutenant  Colonel  Robbins  to  the  TAC/CC  in 
1976  summarized  the  key  points  of  IOT&E:  “1)  No  dedicated 
IOT&E  flights,  2)  Fulfill  IOT&E  objectives  to  the  extent  pos¬ 
sible  on  Air  Force  Preliminary  Evaluation/Contractor  Test 
missions,  [and]  3)  Emphasis  on  basic  aircraft  handling 
qualities,  flying  and  operational  qualities.”37 

It  is  remarkable  that  the  F-15A’s  IOT&E  was  an  add-on 
to  the  developmental  tests  and  that  it  was  very  limited  in 
scope.  Today  we  expect  IOT&E  pilots  to  fly  the  jet  in  the 
same  fashion  as  operational  pilots  will  fly  the  aircraft.  The 
same  was  true  in  1976  but  in  a  much  more  limited  sense. 
The  following  test  events  for  IOT&E  demonstrate  the  sim¬ 
plistic  nature  of  this  evaluation  phase  at  that  time: 

1)  Preflight:  Start,  taxi,  ground  operations 

2)  Takeoff:  Flying  qualities,  gear  retraction 

3)  Climb  and  en  route:  Cockpit  visibility,  formation 

4)  Maneuvering:  Flying  qualities,  buffet,  stall,  tracking, 
performance,  offensive  and  defensive  maneuvers 

5)  Recovery:  Approach,  landing,  rollout 

6)  Mission  equipment:  One-man  operability38 

The  list  of  test  items  is  suggestive.  Did  taxi  and  takeoff 
receive  as  much  evaluation  as  offensive  and  defensive  ma¬ 
neuvers?  How  complex  was  this  phase  of  the  testing?  It 
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almost  seems  like  an  afterthought  to  the  F-15’s  test  process 
for  two  reasons.  First,  the  USAF  Historical  Research 
Agency  at  Maxwell  AFB,  Alabama,  contained  no  IOT&E  re¬ 
port.  Second  are  the  comments  of  Lieutenant  General 
Starbird,  the  deputy  director  for  T&E,  Office  of  the  Direc¬ 
tor  of  Defense  Research  and  Engineering  (DDR&E) ,  to  re¬ 
searcher  Colonel  Mclnerney  in  1973  that  “he  would  have 
liked  to  see  a  ‘heavier’  IOT&E  phase  and  the  Tactical  Elec¬ 
tronic  Warfare  System  was  not  available  for  testing  prior  to 
the  production  decision.”39  The  research  here  implies  that 
the  F-15A’s  IOT&E  was  of  limited  utility  because  of  its  add¬ 
on  nature,  the  limited  focus  of  the  testing,  the  possible 
nonavailability  of  the  results,  and  the  comments  of  leader¬ 
ship  at  the  time.  The  scope  and  nature  of  the  next  phase 
of  testing  also  tend  to  confirm  this  hypothesis. 

Follow-On  Operational  Test  and  Evaluation 

Follow-on  OT&E  (FOT&E)  was  conducted  at  Luke  AFB, 
Arizona,  between  March  1975  and  July  1976.  The  test  re¬ 
port’s  executive  summary  notes  that  the  “purpose  of  the 
evaluation  was  to  verify  the  operational  effectiveness  and 
operational  suitability  of  the  production  F-15A  weapon  sys¬ 
tem.”40  It  appears  that  FOT&E  was  the  major  evaluation 
used  to  prepare  the  F-15A  for  operational  service.  During 
the  test,  live  weapons  were  employed;  maintenance  was 
evaluated;  operational  deficiencies  were  found  requiring 
immediate  corrective  action;  and  specific  operational  and 
planning  factors  were  determined,  such  as  cost  of  owner¬ 
ship.41  The  specific  test  objectives  to  measure  operational 
effectiveness  were  more  detailed  than  in  IOT&E: 

1)  Verify  the  operational  effectiveness  of  the  F-15A  air¬ 
craft  in  its  primaiy  role  as  an  air -superiority  fighter. 

2)  Verify  the  operational  effectiveness  of  the  F-15A  air¬ 
craft  in  its  secondary  role  as  an  air-to-ground 
weapons  system. 

3)  Verify  the  operational  effectiveness  of  the  F-15A  air¬ 
craft  during  routine  phases  of  flight. 

4)  Evaluate  the  reliability  and  maintainability  charac¬ 
teristics  of  the  F-15A  and  assess  system  availability 
and  logistical  supportability. 
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5)  Estimate  the  probability  that  the  F-15A  will  success¬ 
fully  complete  representative  missions. 

6)  Determine  manpower  requirements  for  operation 
and  support  of  the  F- 1 5A. 

7)  Evaluate  the  cost  of  ownership  for  the  F-15A.42 

As  the  last  step  in  T&E  for  the  F- 1 5A,  a  number  of  as¬ 
pects  come  to  light.  This  test  was  completed  four  months 
before  IOC  at  Langley  and  was  concurrent  with  the  pilot¬ 
training  programs  for  the  Langley  IOC  pilots.  It  also  went 
into  considerable  depth  of  evaluation  versus  the  IOT&E. 
FOT&E  utilized  1,111  F-15A  sorties  and  approximately 
900  support  sorties.43  Also,  over  1,000  bombs  were  dropped, 
demonstrating  the  secondary  capability  of  the  F-15A.44 
However,  the  weapons  systems  were  not  fully  tested.  The 
AIM-9L,  the  major  weapons  change  from  the  F-4  Phantom, 
was  not  available  for  FOT&E.  A  recommendation  from  the 
report  was  that  “incorporation  of  the  AIM-9L  missile  should 
be  expedited,”45  and,  indeed,  this  weapon  would  not  be 
fired  from  the  F-15A  until  the  end  of  1977. 46  While  FOT&E 
was  more  detailed  than  IOT&E,  the  research  conclusion 
here  is  that  the  F-15A  weapons  system  was  not  tested  in 
its  entirety  until  nearly  one  year  after  IOC. 

Illustrative  conclusions  can  be  drawn  from  this  review. 
The  F-15A’s  T&E  process  was  remarkable  and  produced  a 
superior  flying  fighter.  But  hindsight  from  2005  yields 
some  interesting  observations.  IOT&E  was  added  onto  de¬ 
velopmental  testing,  and  FOT&E  was  not  completed  until 
four  months  prior  to  IOC.  Given  these  developments,  how 
much  information,  in  a  completed  form,  was  available  to 
Langley’s  IOC  pilots?  The  limited  nature  of  operational 
testing  and  the  short  time  from  its  completion  to  IOC  must 
have  posed  difficulty  in  transferring  lessons  learned  to  IOC 
pilots.  Certainly,  this  must  have  resulted  in  the  F-15A  at 
IOC  being  less  of  a  weapons  system  than  it  would  be  in  a 
matter  of  one  or  two  years.  The  question  remains — should 
tactics,  techniques,  and  procedures  be  developed  after  IOC 
or  before?  With  the  F- 1 5A,  they  seem  to  have  been  devel¬ 
oped  after  IOC. 

The  F-15A’s  T&E  process  foreshadows  the  current  system 
but  without  the  detail  and  time  needed  for  a  complete  and 
timely  report.  Further  evidence  of  this  lies  in  the  execution 
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of  an  additional  F-15A  test  called  Air  Intercept  Missile 
Evaluation  (AIMVAL)/Air  Combat  Evaluation  (ACEVAL). 
AIMVAL/ACEVAL  culminated  in  June-November  1977, 
half  a  year  after  IOC.47  The  testing  was  flown  on  the  Nellis 
AFB,  Nevada,  test  ranges  with  F-15s  and  F-14s  fighting 
F-5Es  in  scenarios  illustrative  of  the  projected  scenario  of 
visual  air  combat  in  the  1985  time  frame.48  And  the  results 
of  the  AIMVAL/ACEVAL  tests?  Some  would  argue,  as  this 
author  and  F-15C  instructor  pilot  (IP)  does,  that  the  tests 
in  1977  were  the  first  evaluations  of  the  F-15A’s  weapons 
system  in  the  configuration  and  with  the  weapons  that  it 
was  designed  to  use.  However,  many  saw  the  actual  results 
as  problematic: 

The  AIMVAL/ACEVAL  exercises  were  badly  flawed  and  very  expen¬ 
sive.  The  flaws  .  .  .  had  to  do  with  an  unrealistic  scenario,  incorrect 
assumptions  on  missile  capabilities,  and  somewhat  arbitrary  rules 
on  equipment  carriage.  .  .  .  Due  to  the  aggressive  and  competitive 
nature  of  fighter  pilots,  this  quickly  changed  from  a  good-natured 
evaluation  of  hypothetical  missile  concepts  to  as  close  as  one  can 
get  to  combat  without  actually  firing  weapons.49 

Clearly,  the  AIMVAL/ACEVAL  test  was  late  and  imperfect, 
but  it  was  the  test  that  was  perhaps  needed  to  close  F-15A 
development  before  IOC  was  declared.  It  serves  to  summarize 
the  F-15A’s  T&E  program — magnificent  but  incomplete 
and  overdue.  The  F-15A’s  T&E  needed  earlier  operational 
involvement  (IOT&E)  and  integration  of  the  AIM-9L  into  the 
aircraft.  Ultimately,  developmental  programs  must  achieve 
a  balance  between  testing  in  the  IOC  configuration  and 
meeting  production  milestones.  Unfortunately,  when  the 
developmental  program  fails  to  meet  milestones,  the  only 
solution  is  either  to  slip  the  IOC  date  or  to  finish  develop¬ 
mental  testing  after  IOC  (as  AIMVAL/ACEVAL  did).  In 
either  case,  the  operational  unit  assumes  the  risk  and 
must  rise  to  the  task  during  preparation  for  IOC.  The  next 
section  evaluates  the  1st  TFW’s  experience  with  the 
F-15A’s  IOC  process. 


F-15A  IOC  at  Langley  AFB, 
October  1976 


The  F-15A’s  IOC  was  the  culmination  of  the  Eagle’s  de¬ 
velopment  process  that  put  the  weapons  system  in  the  war 
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fighter’s  hands.  While  testing  at  Nellis  AFB  would  continue, 
1976  marked  a  shift  in  focus  from  operational  testing  to 
combat/training  operations  in  the  field.  A  number  of  les¬ 
sons  are  noteworthy  regarding  this  stage  of  development. 

The  actual  celebration  of  the  F-15A’s  arrival  at  Langley 
AFB — Eagle  Day  on  9  January  1976 — was  attended  by 
members  of  Congress,  generals,  and  of  course,  the  men 
and  women  of  the  1st  TFW.50  Comments  on  record  from 
visiting  dignitaries  testily  to  the  excitement  associated  with 
the  arrival  of  the  F-15A.  Virginia’s  First  District  congress¬ 
man  concluded  by  saying  that  “every  American  can  breathe 
a  little  easier  when  he  knows  the  Eagle  is  on  the  perch.”51 
The  F-15’s  revolutionary  impact  on  TAC  was  expressed  by 
Gen  Robert  J.  Dixon,  TAC  commander,  as  he  spoke  to  the 
1st  TFW: 

You  will  set  the  pace  for  all  who  will  follow  .  .  .  you  into  the  F-15, 
but,  more  importantly,  you  will  set  the  pace  for  those — like  you — 
who  will  be  first  in  the  AWACS  [Airborne  Warning  and  Control  Sys¬ 
tem],  A-10  and  the  F-16.  You  have  the  .  .  .  responsibility  to  test  the 
validity  of  our  research  and  development  of  our  maintenance  concepts 
and  procedures,  .  .  .  our  tactics  and  training  .  .  .  and  to  set .  .  .  higher 
standards  that  .  .  .  become  the  baseline  for  those  who  follow.52 

During  1975  the  1st  TFW’s  focus  on  beddown  issues  re¬ 
volved  around  three  areas — facilities  development,  wing 
organization,  and  initial  training.53  The  change  from  the 
F-4  to  the  F-15A  necessitated  many  facility  changes,  and 
the  shift  to  line  replaceable  unit  (LRU)  maintenance  and 
intermediate-level  maintenance  necessitated  changes  in 
hangars  and  work  centers  at  Langley.54  A  critical  aspect  of 
facilities  was  the  “need  to  coordinate  .  .  .  [their]  develop¬ 
ment  .  .  .  with  the  flow  of  incoming  aircraft,  equipment, 
and  personnel.”55  Transition  in  1976  from  the  two-seat  F-4 
Phantom  to  the  single-seat  F-15A  Eagle  also  required  a 
change  in  manning,  resulting  in  organizational  changes. 
Such  changes  faced  by  the  1st  TFW  in  1976  due  to  the  loss 
of  the  second  aircrew  were  not  duplicated  in  the  F/A-22’s 
IOC  and  are  not  considered  in  this  research. 

Initial  training  for  both  maintenance  crews  and  pilots 
seems  to  have  been  the  most  critical  issue  for  the  wing  in 
1976.  Obviously,  aircraft  knowledge  and  training  were 
achieved  via  the  testing  process  and  the  Replacement 
Training  Unit  at  Luke  AFB,  Arizona;  they  were  transferred 
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to  the  1st  TFW  by  means  of  written  guidance  as  well  as  the 
likely  transfer  of  experienced  personnel  to  Langley  AFB. 
Pilot  training  at  Luke  AFB  included  the  following  aspects: 

Conversion  training,  F-15,  was  conducted  at  Luke  AFB  by  the 
555th  Tactical  Fighter  Training  Squadron.  The  training  was  divided 
Into  two  phases,  conversion  and  aerial  attack.  IPs  received  a  third 
phase.  .  .  .  Completion  of  the  phases  Is  concurrent  with  an  88  hour 
academic  program  comprising  8  classes.  .  .  .  Sorties  for  line  pilots 
were  lowered  to  21.  .  .  .  Cockpit  Procedural  Trainers  were  used  to 
complete  38  lessons  to  augment  academic  and  flying  training.  .  .  . 

For  simulator  training  .  .  .  aircrews  had  to  go  to  St.  Louis,  MO.56 

But  the  real  training  issue,  which  resided  at  Langley 
AFB,  was  almost  a  catch-22  situation  between  operations 
and  maintenance.  Operations  needed  to  fly,  and  fly  a  lot, 
so  as  to  gain  and  maintain  proficiency  obtained  from  the 
Luke  F-15  course.  But  that  required  jets  on  station  and  a 
steadily  increasing  sortie-production  rate.  Paradoxically, 
the  first  quarter  of  1976  saw  a  limit  placed  on  sortie  pro¬ 
duction:  ‘The  training  of  the  1  TFW  pilots  prior  to  the  wing 
possessing  aircraft  and  slower  than  programmed  aircraft 
delivery  resulted  in  fewer  than  required  sorties  being  avail¬ 
able  this  quarter.  The  wing  was  some  78  sorties  short  of 
meeting  the  minimums  for  Mission  Ready,  Mission  Capable, 
and  Basic  Proficiency  Aircrews.”57  To  mitigate  the  effects, 
the  wing  sent  pilots  to  Holloman  AFB,  New  Mexico,  for  T-38 
sorties  and  also  received  loaner  F-15s  from  Luke  AFB.58 

Looking  at  aircraft  beddown  would  help  in  understand¬ 
ing  the  nature  of  the  sortie-generation  problem.  The  first 
F-15  for  maintenance  training  arrived  in  October  1975, 59 
the  first  aircraft  for  operational  use  arrived  on  18  December 
1975, 60  Eagle  Day  was  9  January  1976,  and  IOC  was  de¬ 
clared  in  October  1976.  As  F-15s  arrived,  sortie  production 
increased  from  26  in  January,  to  93  in  February,  and  to 
183  in  March.61 

But  actual  sortie  production  was  still  beneath  planned 
production.  The  reasons  for  this  shortfall — planning  as¬ 
sumptions,  maintenance,  and  technology — provide  insight 
when  one  considers  sortie  generation  with  new  aircraft. 
Captains  Tom  Lennon  and  Jim  Wray  of  the  1st  TFW  de¬ 
scribe  the  biggest  misconception  regarding  sortie  produc¬ 
tion  as  “overcommitment .  .  .  resulting]  .  .  .  from  excessive 
optimism  based  on  data  from  contractor,  developer  and 
user  test  programs.  Consequently,  operations  in  TAC  started 
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with  an  advertised  capability  of  1. 13  sorties  per  aircraft  per 
day.  .  .  .  Initial  tasking  was  based  on  what  was  thought  to 
be  a  conservative  .9  sortie  production  rate  (SPR).  However, 
in  practice,  a  .61  SPR  was  the  best  achieved,  and  even  then 
only  with  considerable  difficulty.”62  If  Langley  AFB  in  1976 
experienced  a  sortie  rate  60  percent  of  that  planned,  then 
many  pilots  and  maintainers  watched  jets  that  sat  on  the 
flight  line  as  they  fell  further  behind  on  the  path  to  IOC. 

Maintenance  was  the  reason  for  low  sortie  rates  during 
the  final  push  to  IOC.  It  was  further  explained  that  “air¬ 
craft  reliability  in  that  first  year  measured  by  mean  time 
between  failures  (MTBF)  was  less  than  one-half  that  pre¬ 
dicted.  It  took  considerably  longer  to  repair  each  failure, 
and  spare  parts  did  not  cover  the  needs.”63  In  hindsight  it 
is  logical  to  expect  that  repair  time  would  be  longer  with  a 
new  aircraft  and  that  spare  parts  availability  would  limit 
sortie  production.  Like  sortie  rates,  a  greater  percentage  of 
downtime  for  maintenance  must  be  allocated  when  dealing 
with  new  aircraft. 

Finally,  new  technology  was  a  contributing  factor  for  low 
sortie  rates.  The  F-15A  was  the  lead  aircraft  under  the 
maintenance  concept  of  LRU  repair  at  home  station: 

One  of  the  most  painful  lessons  learned  In  Introducing  the  F-15 
was  the  avionics  automated  test  station  area.  The  concept  Included 
a  built-in  (BIT)  system  in  the  aircraft  to  Isolate  failure  to  any  one  of 
99  LRU’s.  The  failed  LRU  was  to  be  removed  and  replaced — or  If  no 
spares  were  available  to  base  supply — removed,  repaired,  and  re¬ 
placed.  The  faulty  unit  was  then  repaired  In  the  base-level  inter¬ 
mediate  shop  where  computer-driven  test  stations  were  used  to 
isolate  the  problem  to  a  single  shop  replaceable  unit  (SRU).64 

This  was  a  great  maintenance  technological  idea  that  con¬ 
tinues  today,  but  in  the  1976  time  frame  the  practical  re¬ 
sult  was  that  while  Langley  could  test  and  repair  92  of  the 
99  F-15  LRUs,  an  average  of  only  48  percent  were  actually 
repaired  at  base  level — the  remainder  went  back  to  the 
contractor  for  repair  because  test  stations  were  down  for 
extended  periods.65  Clearly,  the  LRU  maintenance  idea 
captured  efficiencies  to  decrease  maintenance  downtime, 
but  the  overall  system  then  had  an  unforeseen  Achilles’ 
heel — test-station  failures — that  negatively  affected  IOC. 

Assumptions,  maintenance,  and  technology  combined 
to  create  challenges  for  sortie  production  and,  ultimately, 
training  limitations  for  IOC.  In  a  surprising  move,  the  1st 


18  F-15A  VERSUS  F/A-22 


TFWs  leadership  selected  a  unique  method  for  preparing 
for  IOC — deployments.66  The  1st  TFW’s  F-15As  partici¬ 
pated  in  four  30-day  deployments  during  1976. 67  Seven¬ 
teen  days  after  the  first  squadron  had  received  24  aircraft, 
the  27th  TFS  deployed  to  Nellis  AFB  for  Red  Flag  training.68 
While  assuming  risk  by  taking  a  new  weapons  system 
away  from  home  station,  the  decision  to  conduct  operations 
on  the  road  was  crafted  around  three  focused  reasons: 

First,  the  F-15  represented  a  much  larger  advance  in  air  superiority 
capability  than  had  been  previously  experienced  by  those  associ¬ 
ated  with  the  F-15  program  and  it  was  obvious  that  tactics  to  ex¬ 
ploit  these  capabilities  were  lagging.  .  .  .  Secondly,  since  training 
sorties  were  the  most  precious  resource  available,  it  made  sense  to 
go  where  sorties  were  the  most  productive.  The  third  reason  is 
somewhat  more  controversial  but,  in  the  opinion  of  1st  TFW  leaders, 
the  F-15  support  system  was  geared  in  attitudes  and  pace  to  the 
development  and  training  cycle  rather  than  to  operational  demands. 
Early  deployments  produced  beneficial  pressures  to  shift  gears  and 
the  support  system  responded.69 

The  1st  TFW’s  deployment  to  Nellis  AFB  was  the  catalyst 
that  overcame  the  system’s  early  shortcomings.  How  de¬ 
ployed  operations  could  correct  problems  with  mainte¬ 
nance  and  training  is  a  significant  point.  By  replicating 
combat  conditions  in  a  training  environment,  the  Red  Flag 
deployment  placed  an  artificial  “need”  for  the  F-15A 
weapons  system  to  perform — the  “beneficial  pressures” 
that  Lennon  and  Wray  point  out.  This  aspect  of  the  F- 1 5A’s 
IOC  points  to  the  power  of  leadership:  the  1st  TFW  should 
have  been  able  to  perform  better  at  home  station,  yet 
morale  and  unit  pride  combined  with  demanding  and  real¬ 
istic  off-station  training,  became  keys  to  Langley’s  suc¬ 
cessful  IOC. 

In  summary,  facilities,  training  (operations  and  mainte¬ 
nance),  sortie  production  (assumptions  and  technology), 
and  deployability  seem  to  be  focus  areas  for  the  F-15A’s 
IOC.  Training  and  sortie  production  were  the  limiting  fac¬ 
tors,  while  deployments  were  used  as  a  solution.  Was  the 
F-15A’s  IOC  a  success  in  October  1976?  The  USAF  had 
one  squadron  IOC  with  a  second  to  follow  in  three  months. 
Even  more  telling  was  the  wing’s  ability  to  train  and  gen¬ 
erate  a  squadron  of  jets  bound  for  Bitburg  Air  Base,  West 
Germany,  by  April  1977. 70  Clearly,  the  F-15A’s  develop¬ 
ment  process  put  “iron  on  the  ramp,”  with  an  ability  to  de- 
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ploy  and  train  soon  after  delivery.  The  F-15A’s  T&E  was  late 
and  incomplete,  but  with  the  exception  of  the  AIM-9L  missile, 
the  combatant  commander  (COCOM)  who  would  use  the 
F-15A  at  IOC  would  have  a  significant  improvement  over 
the  F-4  Phantom.  That  successful  IOC  was  a  result  of  the 
challenges  met  and  overcome  by  the  1st  TFW.  Would  these 
challenges  be  repeated  in  the  F/A-22’s  IOC  29  years  later? 


F/A-22  Test  and  Evaluation  Process 

The  F /A-22  Raptor  is  a  great  leap  in  aircraft  technology. 
It  is  a  Cold  War-legacy  weapons  system  that  has  under¬ 
gone  careful  scrutiny  and  emerged  as  a  needed  paradigm 
shift  for  the  air  and  space  expeditionary  force  (AEF)  of  the 
twenty-first  century.  The  Cold  War’s  end  and  the  review  of 
the  aircraft’s  need  have  combined  to  forge  the  F/A-22  into 
a  complex  and  comprehensive  developmental  program: 

The  overall  development  goal  for  the  F/A-22  is  to  achieve  a  balance 
between  performance,  survivability,  reliability,  maintainability,  and 
affordability.  The  F /A-22  must  have  proven  lethality  to  ensure  first 
look,  first  kill  in  all  operational  environments,  and  maneuverability 
and  acceleration  to  ensure  superiority  over  any  known  or  predicted 
threat  in  the  close-in  fight.  It  must  have  the  survivability  to  conduct 
its  air  superiority  missions  over  enemy  territory.  It  will  do  this  through 
a  balanced  combination  of  supersonic  cruise  (without  afterburner), 
reduced  signatures,  tailored  countermeasures,  and  maneuverability. 
The  F/A-22  systems  must  provide  the  pilot  significantly  improved 
beyond  visual  range  situational  awareness  using  its  highly  inte¬ 
grated  offensive  and  defensive  systems.  The  Air  Force  will  field  the 
F /A-22  with  current  generation  weapons,  but  it  must  be  capable  of 
taking  advantage  of  improved  and  follow-on  weapons.71 

The  F/A-22’s  road  to  IOC  is  very  similar  to  the  F-15A’s 
but  with  a  few  marked  differences  that  demonstrate  the  re¬ 
finement  of  the  USAF’s  T&E  process.  The  F/A-22’s  pro¬ 
gram  is  under  careful  DOD  oversight  but  still  attempts  to 
give  the  war  fighter  a  capable  fighter  at  the  outset  of  IOC. 
The  milestones — decision  points — of  the  F/A-22’s  produc¬ 
tion  and  testing  phases  serve  several  vital  functions  (fig.  2). 
First,  they  serve  to  keep  the  USAF,  Office  of  the  Secretary 
of  Defense  (OSD),  and  Congress  informed  as  to  program 
status.  Second,  they  allow  decision  makers  the  means  to 
ensure  that  the  weapons  system  is  meeting  expectations 
during  development.  This  means  that  the  aircraft  pur¬ 
chased  not  only  meets  the  ORD  but  also  keeps  the  con- 
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Figure  2.  F/A-22  milestones.  (From  Lt  Col  Scott  Ruflin,  HQ  ACC  DR-F/A-22,  Langley  AFB,  VA,  Power¬ 
Point  briefing  and  notes,  subject:  F/A-22  Raptor,  18  June  2004.) 
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tractor  from  having  to  react  to  changes  to  contracts  or  ex¬ 
pectations  by  the  military.  Figure  2  demonstrates  how  the 
F/A-22  proceeded  from  a  first-flight  aircraft  to  a  prototype 
and  then  to  a  high-rate  production  aircraft  off  the  assembly 
line.  This  process  yields  the  following  terms:  validation  air¬ 
craft  (YF-22A),  engineering  and  manufacturing  development 
(EMD)  vehicles,  production  representative  test  vehicles 
(PRTV),  low-rate  initial  production  (LRIP)  vehicles,  and,  fi¬ 
nally,  full-rate  production  (FRP)  vehicles.  These  aircraft 
combine  to  yield  a  T&E  fleet  used  for  developmental  and 
operational  testing. 

Like  the  F-15A,  the  F/A-22’s  test  program  was  divided 
between  developmental  and  operational  testing.  Key  agencies 
are  the  F/A-22  SPO  (developer).  Air  Combat  Command  (ACC) 
(operators,  replaced  TAC  in  early  1990s),  and  Air  Force  Op¬ 
erational  Test  and  Evaluation  Center  (AFOTEC)  (evaluators, 
replaced  the  AFTEC).  Interestingly,  the  ties  between  these 
agencies  occur  early  in  the  process.  The  F/A-22’s  first 
flight  was  in  1990,  AFOTEC  was  involved  with  development 
from  1996,  and  operational  testing  did  not  begin  until 
2003.  AFOTEC’s  final  report  on  the  F-22  states  that  the 
“AFOTEC  team  has  been  fully  integrated  with  the  develop¬ 
ment  effort  from  early  on  to  ensure  an  in-depth  under¬ 
standing  and  comprehensive  insight  in  preparing  for  the 
OA  (operational  assessment).  Since  1996,  AFOTEC  has 
supported  virtually  all  F-22  IPT  (integrated  process  teams) 
and  working  groups.”72 

Another  “F-22”  test  program  necessity  was  to  determine 
how  best  to  evaluate  the  fighter  and  attack  missions  of  the 
F/A-22:  “The  [F/A-22’s]  .  .  .  IOT&E  will  focus  on  the  ‘F’  or 
air  superiority  fighter  capability  of  the  F/A-22.  .  .  .  FOT&E, 
currently  scheduled  to  start  in  June  2005,  will  focus  on 
closing  out  unresolved  and  deferred  IOT&E  issues  and  on 
the  A’  or  attack  capability.”73  Just  as  the  F-15A  “morphed” 
into  an  attack  version — the  F- 1 5E  Strike  Eagle — so  did  the 
F/A-22  shed  its  single-mission  facade  and  became  an  air- 
and-space  dominance  vehicle.74 

Developmental  Test  and  Evaluation 

Like  the  F-15A,  F/A-22  DT&E  remained  a  contractor  and 
USAF  Combined  Test  Center  mission.  The  test  emphasis 
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was  on  airframe  certification  and  EMD  process  certifica¬ 
tion  and  modification.  The  basics  of  ORD  validation  were 
accomplished  in  this  phase.  The  F/A-22's  DT&E  was  a 
critical  step  in  the  testing  of  a  new  weapons  system,  but  it 
did  have  limitations  for  the  operational  pilot.  AFOTEC’s 
final  report  on  the  F-22  weapons  system  also  indicates 
that  “it  is  important  to  note  that  this  stage  of  DT&E  in¬ 
cludes  virtually  no  mission  avionics  testing  because  those 
capabilities  have  yet  to  be  included  in  the  test  fleet.  As  a 
result,  data  for  Operational  Assessments  includes  results 
of  F-22  testing  from  a  variety  of  sources  including  various 
laboratories,  models  and  simulations,  and  the  Flying  Test 
Bed  (FTB).”75 

While  DT&E  had  a  distinctly  contractor/test-pilot  flavor, 
the  F/A-22’s  DT&E  did  attempt  to  look  forward  to  opera¬ 
tional  testing  by  incorporating  an  operational  pilot.  Lt  Col 
David  M.  “Doc”  Nelson,  a  graduate  of  the  USAF  Weapons 
School  and  Test  Pilot  School,  flew  the  F/A-22  during  DT&E 
to  assess  its  Military  Utility  Testing  (MUT),76  designed  to 
provide  “an  unprecedented  degree  of  operational  insight 
into  developmental  testing  with  corresponding  benefits  to 
both  phases  of  testing.”77  Colonel  Nelson’s  evaluation  of 
the  F/A-22’s  ability  to  be  ready  for  operational  testing  re¬ 
viewed  such  areas  as  signature,  speed,  flying  qualities,  air- 
refueling  handling  qualities,  agility,  and  so  forth.78  How¬ 
ever,  “MUT  was  not  completed  until  one  month  prior  to 
dedicated  operational  test.  This  left  no  time  to  implement 
any  changes  based  on  MUT  findings.”79 

This  brief  look  at  the  F/A-22’s  DT&E  yields  three  obser¬ 
vations.  First,  as  for  the  F-15A,  DT&E  remained  focused 
on  airframe  and  manufacturing  development.  Second,  the 
need  to  “look  ahead”  to  operational  testing  is  recognized 
(via  MUT)  yet  remained  difficult  to  accomplish.  Finally,  the 
need  to  have  cooperative  and  overlapping  T&E  (developer 
to  operator)  is  a  necessary  part  of  the  acquisition  process. 

Operational  Assessment 

The  Defense  Acquisition  Board  (DAB)  established  the  re¬ 
quirement  for  an  operational  assessment  to  support  the 
decision  to  proceed  with  F/A-22  LRIP  in  August  2001. 80 
The  F/A-22’s  OA  ran  from  January  1998  to  December 
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2000  and,  using  the  results  of  the  DT&E  missions,  ulti¬ 
mately  supported  the  certification  of  system  readiness  for 
operational  testing.81  In  supporting  the  LRIP  decision,  the 
OA  evaluated  the  F/A-22  against  “the  user’s  concept  of  op¬ 
erations  (CONOPS),  the  ORD,  and  joint  employment  con¬ 
cepts  to  define  F-22  system  requirements  and  the  environ¬ 
ment  in  which  it  is  expected  to  operate.”82  The  OA  also 
looked  at  the  challenges  or  issues  with  further  operational 
testing  and  considered  the  operational  environment  that 
the  F/A-22  would  operate  in.  Specifically,  the  OA  contains 
the  critical  operational  issues  (COI)  and  key  performance 
parameters  (KPP)  (table  1): 


Table  1.  F/A-22  critical  operational  issues  and  key  perfor¬ 
mance  parameters 


F-22  Critical  Operational  Issues 

1.  Is  the  F-22  support  package  required  to  meet  sortie  surge  rates  in 
a  global  environment  deployable  with  the  authorized  airlift? 

2.  Can  the  F-22  execute  the  counterair  mission  in  the  intended  op¬ 
erating  environment? 

3.  Is  the  F-22  more  survivable  operating  in  the  intended  surface-to- 
air  and  air-to-air  environment? 

4.  Will  the  F-22  meet  the  required  sortie  surge  rate? 

5.  Can  the  F-22  deliver  air-to-surface  munitions  (JDAM  [Joint  Direct 
Attack  Munition]  1 000)  during  day,  night,  and  all  weather  operations? 

F-22  Key  Performance  Parameters 

Radar  Cross  Section 
Supercruise 
Maneuver 
Acceleration 

Radar  Detection  Range  Data 

Combat  Radius 

Payload 

Mean  Time  between  Maintenance 
Sortie  Generation  Rate 
Independent  Airlift 


Reprinted  from  AFOTEC  Final  Report  00-27,  F-22  Weapons  System  Operational  Assessment 
for  LRIP  (Kirtland  AFB,  NM:  Air  Force  Operational  Test  and  Evaluation  Center,  2000),  5. 


The  COI/KPP  product  frames  the  entire  OT&E  process 
to  produce  meaningful  results.  Note  that  COIs  one  through 
four  were  tested  in  IOT&E,  while  COI  five  is  held  for 
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FOT&E.  (“As  directed  by  SAF/AQ  in  the  Program  Manage¬ 
ment  Document,  AFOTEC  will  defer  this  COI  until  Joint 
Direct  Attack  Munition  development  has  been  completed.”)83 

Combined  Developmental  Test/Operational  Test 

Like  the  F/A-22’s  OA  for  LR1P,  another  aspect  of  F /A-22 
testing  that  is  distinct  from  F- 1 5A  testing  is  the  combined 
developmental  test  (DT)  and  operational  test  (OT)  relation¬ 
ship.  In  the  F-15A,  IOT&E  was  an  addition  to  the  devel¬ 
oper’s/contractor’s  testing.  With  the  F/A-22  the  relation¬ 
ship  is  formal,  fulfills  a  specific  purpose,  and  meets  Title 
10  US  Code:  “The  term  ‘combined  testing’  means  testing 
conducted  by  the  developmental  and  operational  testers 
when,  because  of  cost,  schedule,  or  test  item  availability, 
they  must  share  test  facilities,  data  or  resources.  Com¬ 
bined  DT/OT  serves  as  a  bridge  between  the  DT&E  and 
IOT&E  test  programs.”84 

The  major  activities  of  combined  DT/OT  testing  are  sen¬ 
sor  integration,  offensive-weapons  integration,  survivability, 
suitability,  climatic  testing,  and  MUT.85  Combined  DT/OT 
testing  meets  the  Title  10  legal  requirements  to  evaluate 
separately  but  still  achieve  economies.  The  beauty  of  the 
combined  DT/OT  testing,  when  matched  with  the  OA,  is 
that  it  better  prepares  the  evaluators  to  provide  a  superior 
product  (in  the  form  of  reports  and  the  actual  weapons 
system)  to  the  operator  at  IOC.  Formal  DT/OT  integration 
is  necessary  to  reduce  cost,  minimize  duplication  of  effort, 
and  push  operational  considerations  further  forward  into 
the  EMD  process;  it  also  clearly  defines  necessary  program 
separations  demanded  by  Title  10:  “DT  and  OT  may  share 
data  and  resources;  however,  OT  evaluators  must  evaluate 
and  report  data  independently.”86  Finally,  it  is  useful  to 
have  some  familiarity  with  the  timelines  for  developmental 
and  operational  testing  (fig.  3). 

F/A-22  to  F-15C  Comparison  Testing 

An  interesting  aspect  of  US  fighter  development  is  that 
the  T&E  process  is  not  set  in  stone.  The  DOD  can  inject 
changes  to  the  process  to  meet  political,  budgetary,  or 
oversight  requirements.  An  example  of  this  occurred  in 
1977  with  the  F-15A’s  program  when  the  AIMVAL/ACEVAL 
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Calendar  Year 


Figure  3.  Timelines  for  developmental  testing/operational 
testing.  Air  Force  Operational  Test  and  Evaluation  Center,  F-22 
Combined  DT/OT  Requirements  Document  (Kirtland  AFB,  NM, 
2000),  1-3. 


test  was  guided  by  the  director  of  defense  research  and  en¬ 
gineering  as  a  joint  Navy /Air  Force  program  with  the  Navy 
designated  as  executive  service.87  AIMVAL/ACEVAL  looked 
at  air  combat  in  1985,  and  “the  results  identified  advan¬ 
tages  and  deficiencies  of  current  air  superiority  aircraft  in 
a  fighter  sweep  scenario  and  suggested  changes  in  re¬ 
search  and  development  efforts,  procurement,  training 
and  tactics  to  meet  a  forecast  highly  lethal,  all-aspect 
threat  in  the  visual  arena.”88 

In  the  case  of  the  F/A-22,  a  comparison  between  the 
F-15C  and  its  replacement,  the  F/A-22,  was  directed  via 
an  acquisition  decision  memorandum  (ADM)  in  1992  to 
compare  effectiveness  as  a  prerequisite  for  passing  the 
FRP  decision.89  AFOTEC  was  directed  “to  compare  the  mis¬ 
sion  effectiveness  of  the  F /A-22  to  the  F- 1 5C  at  the  Nellis 
Test  and  Training  Range  (NTTR)  with  a  criterion  of  ‘two 
times  better’  under  identical  open-air  conditions.”90 

To  meet  the  ADM’s  requirements,  F-15C  and  F/A-22 
aircraft  flew  identical  missions  with  specific  methods  of 
evaluation  and  performance  to  prove  the  F/A-22’s  two- 
times-better  superiority.  These  missions  are  representative 
of  current  air-to-air  doctrine  and  included  force  protection; 
high-value  airborne-asset  attack;  and  first-look,  first-kill 
(FLFK)  events.91  This  final  event  is  unique  in  that  it  focuses 
on  the  prime  reason  for  the  F/A-22’s  superiority  and  also 
enables  the  evaluators  to  focus  on  weapons-system  avionics: 
“ACC’s  top-level  ORD  requirement  for  the  F /A-22  is  that  it 
achieve  a  first  look,  first  shot,  and  first  kill.  Therefore,  the 
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purpose  of  the  FLFK  is  to  determine  the  extent  to  which 
the  F/A-22  and  F-15C  have  an  opportunity  to  detect,  en¬ 
gage,  and  launch  a  missile  that  reaches  endgame  before 
the  Red  AI  [airborne  interceptor]  has  an  opportunity  to 
launch  a  missile.”92 

Phase  One  Operational  Test  and  Evaluation 

As  with  comparison  testing  (CT),  Phase  One  OT&E 
(OT&E[l])  is  an  ADM-directed  adjustment  to  the  F/A-22’s 
test  program  that  demonstrated  flexibility  and  a  committed 
focus  to  operational  evaluation.  While  OT&E(l)  baselined 
the  F/A-22’s  performance,  it  also  incorporated  open-air 
missions  and  simulator  sorties  at  the  Marietta  Air  Combat 
Simulator  (ACS)  facility.93  The  point  to  OT&E(l),  however, 
is  not  the  specific  tests  or  number  of  missions  flown,  but 
the  attitude  that  it  demonstrates.  OT&E(l)  was  the  intro¬ 
duction,  or  warm-up,  for  IOT&E.  As  such,  it  demonstrated 
a  commitment  to  make  IOT&E  as  real,  challenging,  and 
“evaluate-able”  as  possible:  “The  IOT&E  pilots  will  fly  open- 
air  OT&E(l)  and  IOT&E  trials  at  the  Edwards  AFB  Ranges 
and  NTTR  and  then  ‘fly’  240  IOT&E  trials  in  the  ACS.  It  is 
critical  that  they  stabilize  at  a  high  level  of  F/A-22  flying 
proficiency  and  also  be  highly  proficient  in  F /A-22  tactical 
employment  before  starting  OT&E(l)  and  IOT&E.”94 

As  a  warm-up  for  IOT&E,  OT&E(l)  emphasized  that  the 
F/A-22’s  IOT&E  was  more  complex  than  the  F-15A’s  (the 
“add-on”  to  DT).  It  was  more  detailed  in  scope — more  opera¬ 
tional  and  tactical  measures  of  effectiveness  than  the  F-  15A’s 
evaluation  of  offensive  and  defensive  maneuvers — and 
demonstrated  a  desire  to  train  the  evaluators  to  a  higher 
standard  than  the  F-15’s  IOT&E  or  even  its  FOT&E.  The 
F/A-22’s  pilot,  maintenance,  and  test-support  personnel 
were  all  trained  via  OT&E(l)  before  IOT&E.95  OT&E(l)  also 
established  a  commitment  to  enter  testing  when  ready. 
AFOTEC  calls  this  an  active  ready-to-test  effort,  which  in¬ 
cludes  “quarterly  reviews  of  all  issues  by  AFOTEC/CC,  Air 
Force  Flight  Test  Center  Commander,  ACC  Director  of  Re¬ 
quirements,  AFOTEC  Director  of  Operational  Test  and 
Evaluation,  the  F/A-22  SPO  Director,  and  others  as  ap¬ 
propriate.”96  The  mission  of  OA,  CT,  combined  DT/OT,  and 
OT&E(l)  was  to  pave  the  way  towards  a  smooth  and  effec- 
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tive  initial  operational  test  of  the  F/A-22.  This  represents 
a  major  change,  even  improvement,  over  the  F-15A’s  T&E. 

Initial  Operational  Test  and  Evaluation 

IOT&E  was  conducted  between  April  and  August  2004.97 
The  test  report  was  not  available  at  the  time  this  research 
paper  was  written.  Missions  were  similar  to  OT&E(l),  with 
some  as  large  as  four  F/A-22s  against  eight  threat  air¬ 
craft.98  During  the  test,  efforts  were  made  to  ensure  that 
the  test  aircraft  were  configured  similarly  to  production 
aircraft:  “This  includes  efforts  to  eliminate  all  production 
representative  differences  with  respect  to  effectiveness, 
such  as  mission  planning,  systems  integration,  employ¬ 
ment,  and  interoperability,  as  well  as  with  respect  to  suit¬ 
ability  such  as  those  that  affect  reliability,  maintainability, 
and  availability.”99 

However,  the  aircraft  available  for  IOT&E  were  not  “full- 
up.”  That  is,  they  had  limitations  to  their  flight  envelope — 
the  program  imposed  aircraft  operating  limitations  that  were 
active  through  IOT&E.100  But  again,  these  restrictions  were 
not  expected  to  complicate  or  invalidate  IOT&E  results  as 
the  impact  was  accountable  by  the  pilots,  and  the  produc¬ 
tion  aircraft  will  have  improved  G  and  airspeed  capabilities 
over  the  IOT&E  aircraft.101 

Follow-on  Operational  Test  and  Evaluation 

FOT&E  is  the  final  step  in  T&E  for  the  F/A-22  en  route 
to  IOC  at  Langley  AFB  in  December  2005.  FOT&E  began 
on  29  August  2005:  “FOT&E  will  focus  on  air-to-ground 
attack  and  re-testing  selected  areas  and  capabilities  found 
either  before  or  during  IOT&E  to  be  deficient.  Also,  AFOTEC 
will  test  and  evaluate  those  combat  capabilities  that  are 
not  released  until  FRP,  yet  still  required  for  full  operational 
capability.  Deferred  testing  includes:  JDAM,  ferry  mission 
demonstration,  and  pilot  training  system.”102 

It  is  notable  that  FOT&E  begins  four  months  before  Lang¬ 
ley  declares  IOC;  the  inferable  impact  is  that  any  critical 
lessons  learned  from  FOT&E  could  be  simultaneously 
passed  to  the  operators  as  they  build  up  to  IOC.  But  while 
lessons  learned  might  be  passed,  changes  to  the  aircraft  or 
its  avionics  advised  by  FOT&E  will  not  be  possible.  In  ad- 
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dition,  FOT&E  will  compete  with  IOC  buildup  for  sortie 
generation  and  maintenance  priorities.  Unfortunately,  as 
was  the  F-15A’s  FOT&E,  the  F/A-22’s  is  late,  relative  to 
IOC.  However,  it  will  be  “complete”  in  that  the  weapons  and 
avionics  in  FOT&E  match  those  of  the  IOC  aircraft. 

Summary 

The  various  phases  of  T&E  for  the  F/A-22  combine  to 
form  an  interesting  picture  of  the  overall  process  that  is 
easily  compared  to  the  F-15A’s  experience.  The  F-15A’s  T&E 
proceeded  from  contractor,  to  USAF,  to  operational  testing 
(add-on  initial  and  then  follow-on  during  the  buildup  to  IOC 
at  Langley  AFB),  and  finally  to  AIMVAL/ACEVAL  testing 
after  IOC  to  satisfy  questions  from  oversight  organizations. 
The  process  fielded  a  capable  weapons  system  at  IOC,  but 
this  researcher’s  overall  impression  of  the  testing  was  that 
it  was  incomplete  and  late. 

By  the  time  the  F /A-22  was  developed,  the  process  had 
changed.  The  program  was  milestone-dominated  to  support 
production  decisions,  and  in  the  case  of  the  OA  for  LRIP 
decision,  data  from  developmental  testing  was  used  for  the 
OA.  DT&E  began  the  process,  economies  were  achieved  to 
allow  combined  DT / OT  testing,  and  operational  testing  oc¬ 
curred  in  the  familiar  forms  of  initial  and  follow-on.  But 
IOT&E(l)  was  added  as  a  warm-up  to  initial  operational 
testing,  and  oversight  guidance  was  honored  by  CT  of  the 
F/A-22  versus  the  F-15C.  As  was  the  case  with  the  F-15A, 
FOT&E  will  be  accomplished  during  the  final  months  of 
IOC  buildup  at  Langley  AFB. 

Is  the  F/A-22’s  T&E  incomplete  and  late,  as  it  was  with 
the  F- 1 5?  The  comparison  is  striking,  yet  the  answer  is  not 
definite — clearly  the  F/A-22’s  process  has  yielded  a  weapons 
system  that  has  been  tested  to  an  operational  standard  far 
exceeding  that  of  the  F-15A.  The  testers  and  evaluators 
have  “done  more”  with  the  F/A-22  than  was  done  with  the 
F-15A.  The  current  test  program  shows  the  unmistakable 
signs  of  the  computer  and  the  information  age — more  detail 
in  the  test  plan,  more  organizational  involvement,  and  a 
greater  depth  of  evaluation  than  was  possible  in  the  1970s. 
While  the  F/A-22’s  COIs/KPPs  do  not  address  every  aspect 
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of  operational  employment,  they  certainly  go  beyond  the 
scope  of  assessment  of  the  F- 1 5A’s  IOT&E  and  FOT&E. 

The  best  conclusion  regarding  the  F/A-22’s  T&E  is  that 
it  used  the  same  basic  model  as  the  F- 1 5A,  but  with  posi¬ 
tive  changes  that  attempted  to  push  operational  testing 
further  forward  in  the  process.  However,  the  same  chal¬ 
lenges  were  met,  and  the  same  timing/deliveiy  issues  were 
repeated.  But  the  USAF’s  experience  of  the  past  29  years — 
along  with  the  concurrent  march  of  technology — has  pro¬ 
duced  an  aircraft  with  more  capability  at  IOC  than  its 
predecessor.  Like  the  F-15A,  the  brunt  of  the  work  for  IOC 
still  falls  to  the  1st  Fighter  Wing  (FW)  at  Langley  AFB.  Per¬ 
haps  the  real  question  after  comparing  the  F- 1 5A  with  the 
F/A-22  is  whether  or  not  the  T&E  process  has  transformed 
during  the  last  29  years.  That  question  will  be  considered 
after  reviewing  the  F/A-22’s  IOC. 


F/A-22  IOC  at  Langley  AFB, 
December  2005 

The  first  F-15s  arrived  at  Langley  AFB  in  December 
1975,  and  so  too  the  F/A-22  Raptor  is  destined  to  become 
IOC  at  Langley  in  December  2005.  The  buildup  to  IOC,  as 
well  as  issues  and  concerns,  is  similar  for  both  aircraft. 
Ironically,  the  comparison  between  IOC  activities  at  Langley 
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AFB  for  each  aircraft  bespeaks  a  30-year  technological 
gap;  unfortunately,  similarity  remains  at  the  core  of  both 
programs.  The  critical  issues  for  the  F/A-22’s  IOC  in  2005 
are  virtually  identical  to  those  in  1976 — facilities,  training 
(operators  and  maintainers) ,  sortie  production  (assump¬ 
tions  and  technology),  and  deployability. 

F/A-22  Facilities  Issues 

Beddown  of  the  F/A-22  at  Langley  AFB  is  closely  moni¬ 
tored  and  planned  at  the  wing  and  major  command  (MAJ- 
COM)  levels  by  means  of  Site  Activation  Task  Force  (SATAF) 
analysis,  program  management  review  (PMR)  coordination, 
and  briefings  to  the  ACC  commander  (COMACC).  The  PMR 
briefings  codify  issues  as  either  red,  yellow,  or  green  with 
respect  to  their  impact  on  IOC  (table  2).  The  chart  indi¬ 
cates  that  planners  have  addressed  construction  issues 
but  that  they  have  not  identified  potential  limitations  to 
IOC.  However,  a  problem  that  was  not  experienced  with  the 
F- 1 5A  at  Langley  is  the  intense  classification  requirements 
associated  with  the  F/A-22  and  the  squadrons,  hangars, 
and  other  maintenance  facilities  needed.  This  problem  is 
similar  to  the  security  issues  associated  with  the  F-117 
and  the  B-2.  While  not  significant,  innovative  solutions  are 
part  and  parcel  of  the  1st  FW’s  leadership  at  Langley  AFB 
during  the  F/A-22  beddown.103 

F/A-22  IOC  Training  Issues 

IOC  calls  for  an  aircraft,  weapons,  a  support  system,  a 
CONOPS  for  combat  employment,  and  the  people  trained 
to  operate  the  weapons  system.  This  presents  a  problem 
since  the  aircraft  has  to  fly  in  order  for  both  pilots  and 
maintainers  to  get  training.  Training  is  accomplished 
through  academic  courses,  transfer  of  lessons  learned 
from  test  and  fighter  training  unit  (FTU)  operations,  main¬ 
tenance  trainers,  and  simulators.  All  of  these  training 
methods  were  in  place  for  the  F-15A,  but  the  march  of  time 
has  definitely  left  its  mark — technology  has  changed  the 
means  of  learning  and  the  degree  of  expertise  required.  By 
way  of  analogy,  replace  the  F/A-22  with  a  2005  Cadillac, 
and  compare  it  to  the  technology,  repair,  and  training 
needed  to  operate  a  1976  Cadillac.  The  computer  is  the 
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Table  2. 1st  Fighter  Wing  facilities  status 

Item 

Current  Status 

Assessment 

Low  Observable  and 
Composite  Repair 
Facility  (LO/CRF) 

Configuration  Review  Board 
(CRB)  has  some  control 
problems  being  corrected 

Yellow 

Flight  Simulator 

Boeing  working  electrical 
concerns  w/their  equipment. 
Estimated  completion  date 
(ECD)-Dec  04 

Yl/Gr 

Flush  Plouse  Repairs 

Flush  Flouse  #2 

ECD-30  Nov  04; 

Flush  Flouse  #1 

E CD-24  Dec  04 

Green 

Weapons  Load  Trainer 
(WLT)  Repair 

Awaiting  Noise/Exhaust 

Data  Report  from  SPO 
to  begin  design 

Yellow 

Missile  Storage 
(Munitions  Storage 

Area  [MSA]) 

Preconstruction  meetings 
complete  by  1  Dec  04 

Green 

Aerospace  Ground 
Equipment  (AGE)/ 
Maintenance  (MX) 
Storage  (MSA) 

Preconstruction  meetings 
complete  by  1  Dec  04 

Green 

Munitions  Assembly 
Conveyor  (MSA) 

Preconstruction  meetings 
complete  by  1  Dec  04 

Green 

Munitions  Storage 

Igloo  (MSA) 

Preconstruction  meetings 
complete  by  1  Dec  04 

Green 

Squadron  Ops  2  &  3 

On  Track 

Green 

Adapted  from  Thomas  Tinsley,  commander,  1  st  Operations  Group,  Langley  AFB,  VA,  PowerPoint 
briefing  and  notes,  subject:  1st  Fighter  Wing  F/A-22  Status  of  Operations,  1  Nov.  2004,  slide  2. 


largest  single  technological  change,  and  the  simulator  is  a 
good  example  of  the  differences  between  the  F-15A’s  and 
the  F/A-22’s  IOC. 

Unlike  the  F-15A’s  pilots,  who  did  not  have  a  simulator 
to  use  at  Langley  AFB  until  late  1976,  the  F/A-22’s  pilots 
have  two  simulators  available.  The  simulator  at  Langley 
AFB  was  open  for  use  in  May  2005,  and  the  ACS  facility  at 
Lockheed  in  Marietta,  Georgia,  is  also  under  contract  for 
pilot  training.104  IOC  pilots  have  had  access  to  simulator 
training  at  their  home  station  for  six  months  prior  to  IOC 
and  to  the  ACS  facility  for  nearly  a  full  year.  But  simulators 
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do  not  replace  real  Hying.  In  terms  of  fighter-pilot  profi¬ 
ciency,  simulators  augment  sorties,  and  it  is  in  this  area 
that  IOC  meets  its  greatest  challenge. 

F/A-22  Sortie  Production 

The  key  to  IOC  from  the  operations  perspective  is  to 
have  proficient  pilots  at  IOC,  and  that  necessitates  the 
need  to  generate  sorties.105  IOC  requires  a  combination  of 
aircraft,  trained  pilots  from  Tyndall  AFB,  Florida  (the  FTU), 
and  proficient  pilots.  As  it  did  for  the  F-15A’s  IOC,  Langley 
AFB  will  receive  loaner  jets  from  Tyndall  AFB  (Luke  in 
1976)  for  maintenance  training  and  operations  flying.106 
The  1st  FW  built  a  plan  that  used  the  best  data  for  aircraft 
delivery  from  the  F/A-22  SPO,  determined  the  proper  level 
of  flying  for  pilot  proficiency  (modeled  on  current  fighter - 
training  standards),  and  made  sortie-generation  assump¬ 
tions  based  on  the  F /A-22’s  performance  at  Tyndall  AFB. 

The  1st  FW  made  some  key  assumptions  that  show  an 
understanding  of  the  lessons  of  the  F-15A.  First,  the  sor¬ 
ties  required  for  pilot  proficiency  are  adjusted  from  those 
used  by  a  fully  operational  aircraft.  Headquarters  ACC’s 
Ready  Aircrew  Program  (RAP)  is  designed  to  codify  the 
training-sortie  rates  needed  for  proficiency  and  thereby 
create  planning  factors  for  sortie  rates  and  the  ability  to 
justify  flying  budgets.107  What  Langley  has  done  is  allow 
for  a  monthly  increase  in  RAP  requirements  leading  to  IOC. 
Pilots  will  receive  increasing  numbers  of  sorties  per  month 
approaching  December  2005  (fig.  4).  The  second  key  as¬ 
sumption  is  expected  sortie  rate.  In  2004  F/A-22s  at  Tyndall 
AFB  flew  on  average  13  times  per  month — or  an  aircraft 
utilization  rate  (UTE)  of  13  sorties  per  month  per  aircraft.108 
Currently,  the  1st  FW’s  leadership  is  programming  for  a 
UTE  that  starts  at  five  sorties  per  month  per  aircraft  and 
increases  to  12  as  IOC  is  approached  (fig.  4). 109  This  allows 
for  maintenance  limitations  (training,  proficiency,  avail¬ 
ability  of  tech-order  data,  etc.)  and  spare-parts  issues 
(availability,  priority,  etc.)  with  the  F/A-22  that  are  similar 
to  those  of  the  F-15A.  The  1st  FW’s  leadership  is  counting 
on  a  learning  curve  that  will  allow  near  parity  with  Tyndall's 
performance  by  IOC.  Failure  to  reach  these  planned  UTE 
rates  provides  significant  issues  for  pilot  proficiency  at 
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Buildup 


D  *■  IOC 
Preparation  ^ 


Minimum  sorties  required  based  upon  phased  plan: 

-  5  sorties  per  month  through  May 

-  8  sorties  per  month  J un-Sep 

-  10  sorties  per  month  (RAP)  Oct-Decfor  18  pilots 
Sorties  available  based  upon  gradual  climb  in  UTE: 

-  Feb  through  Apr-5,  May/Jun-7,  Ju I/Aug- 1 0,  Sep  through  Dec- 12 


Figure  4.  Langley  AFB  RAP  sortie  requirements.  (From  Col 
Thomas  Tinsley,  commander,  1st  Operations  Group,  Langley 
AFB,  VA,  PowerPoint  briefing  and  notes,  subject:  1st  Fighter  Wing 
F/A-22  Status  of  Operations,  1  Nov.  2004,  slide  6.) 


IOC.  If  sortie-production  rates  are  less  than  anticipated, 
then  IOC  will  either  be  declared  with  pilots  with  lower  fly¬ 
ing  proficiency  or  with  fewer  pilots  meeting  RAP-mandated 
levels  of  proficiency. 

Sortie  production  is  critical  to  the  F/A-22’s  IOC.  An  issue 
for  all  phases  of  T&E,  it  will  likely  remain  the  frustrating 
aspect  of  IOC  and  the  area  requiring  planning  flexibility. 
Sortie  production  may  even  be  a  constant  that  cannot  be 
overcome  amongst  all  new  major  weapons  systems. 

Just  as  planning  assumptions  translate  sortie  production 
into  pilot/maintainer  proficiency  at  IOC,  so  will  the  F/A-22’s 
new  technology  affect  IOC.  The  F/A-22  represents  a  tech¬ 
nological  leap  in  combat  performance.  Its  technological  ad¬ 
vances  permeate  the  entire  program  and  create  choke 
points  for  sortie  production.  Two  specific  areas  that  affect 
IOC  at  Langley  AFB  are  modifications  to  the  aircraft  after 
they  leave  the  factory  and  a  specific  system  called  Inte¬ 
grated  Maintenance  Information  System  (IMIS).  While 
these  issues  are  not  “showstoppers”  to  IOC,  they  could  be- 


34  F- 1 5A  VERSUS  F/A-22 


come  the  source  for  lower  sortie  rates  that,  in  turn,  reduce 
pilot  proficiency  at  IOC. 

The  F/A-22’s  production  line  is  maturing  as  the  testing 
proceeds.  EMD  aircraft  serve  to  refine  the  production-line 
process,  and  LRIP  aircraft  allow  for  incorporation  of  modi¬ 
fications  identified  as  needed  during  DT&E  and  OT&E.  The 
actual  incorporation  of  the  needed  modifications  into  the 
fleet  is,  however,  problematic.  Aircraft  are  of  different  lots 
or  production  periods  and  have  slight  differences  requiring 
careful  program  management.  This  problem  is  eradicated 
as  the  aircraft  move  into  FRP  and  as  changes  to  the  air¬ 
frame  or  avionics  are  minimized,  eliminated,  or  made  con¬ 
sistent  between  all  fielded  aircraft.  When  modifications  are 
needed,  changes  can  slow  aircraft  production  and  delivery 
or  require  aircraft  downtime  in  the  field  for  modifications 
to  be  made. 

Three  such  F/A-22  modifications  affect  the  aircraft  for 
Langley  AFB:  Lot  three  Operational  Flight  Program  (OFP) 
software  changes,  infrared  countermeasures  (IRCM) 
shroud  installation,  and  night  air-to-air  refueling  (NAAR) 
lighting  changes.110  The  details  of  these  modifications  are 
beyond  the  scope  of  this  paper,  but  the  solutions  and  im¬ 
pact  on  IOC  are  noteworthy. 

The  NAAR  lighting  modification  is  a  solution  offered  in 
response  to  testing  that  identified  a  difficulty  with  night¬ 
time  refueling  of  the  F/A-22  by  the  KC-135.  The  solution 
is  to  change  the  existing  lighting  and  paint  scheme  around 
the  air -refueling  receptacle.111  The  installation  plan  is  to 
accelerate  the  retrofit  kit  production  and  divide  the  modi¬ 
fication  work  between  the  factory  and  Langley  AFB.112  This 
solution  is  common  in  fighter -aircraft  production  and 
modification  and  uses  the  terms  Group  A  and  Group  B 
modifications.  Group  A  is  the  work  that  can  be  done  im¬ 
mediately  during  production  or  with  parts  on  hand,  while 
Group  B  is  the  portion  of  the  modification  that  can  be 
completed  after  retrofit  kits  are  available  from  the  supplier 
and  are  usually  installed  at  the  operational  location.  This 
Group  A/B  solution  maximizes  the  factory’s  production 
time  while  minimizing  the  downtime  that  the  operational 
site  has  to  use  for  the  modification.  A  response  to  flight 
testing,  IRCM  shroud  modification,  was  also  solved  via  a 
Group  A/B  solution  with  factory  and  operational-site  in- 
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stallations.113  While  this  method  of  fixing  a  deficiency  iden¬ 
tified  in  flight  testing  is  commendable  and  done  most  effi¬ 
ciently,  it  is  not  without  cost. 

The  addition  of  these  two  modifications  alone  requires 
running  a  modification  line  at  Langley  AFB  for  seven  days 
a  week,  using  two  shifts  each,  and  working  10  hours  per 
shift.114  This  kind  of  aircraft  downtime  translates  into  lost 
RAP  sorties  contributing  to  IOC,  greater  demand  on  main¬ 
tenance  at  Langley  AFB,  and  greater  potential  for  delays  as 
aircraft  are  “cracked  open”  to  allow  internal  modifications. 
Realistically,  the  assumptions  graphed  in  figure  4  could 
change  in  real  time  as  the  modifications  are  made. 

The  final  modification  to  the  F/A-22  at  Langley  AFB — 
the  avionics  OFP  change — is  estimated  to  require  more 
modification  downtime  and  is  significant.  Estimated  addi¬ 
tional  downtime  per  jet  is  10  days  beyond  the  other  modi¬ 
fications,  bringing  the  downtime  for  delivered  aircraft  at 
Langley  AFB  to  51  days.115  This  is  a  consequential  issue  as 
the  sortie-production  assumptions  contained  in  the  previ¬ 
ous  figures  reflect  “only  a  one  month  down  time  for  the 
OFP  mod  plan  of  two  jets  per  month.”116  Assuming  that 
aircraft  delivery  significantly  increased  beginning  in  May 
2005,  a  51 -day  downtime  on  each  aircraft  represents  24 
percent  of  the  available  time  to  use  the  aircraft  for  sorties 
before  IOC.  Clearly,  performing  aircraft  modifications  at 
the  operational  site  could  prove  to  be  a  significant  hurdle 
to  achieving  IOC  with  the  training  and  proficiency  desired. 

While  aircraft  modifications  during  IOC  buildup  can  af¬ 
fect  schedules,  the  F/A-22’s  technology  itself  can  create 
friction.  One  such  case  is  the  IMIS,  which 

provides  the  F-22  with  a  paperless  maintenance  environment  and 
Includes  areas  such  as  aircraft  forms,  scheduling,  report  genera¬ 
tion,  [and]  technical  order  data.  ...  By  providing  diagnostic  and  In¬ 
teractive  Electronic  Technical  Manual  (IETM)  data.  F-22  IMIS  re¬ 
duces  man-hours  needed  to  service,  troubleshoot  and  repair  aircraft 
systems.  IMIS  performance  is  critical  to  the  achievement  of  high 
sortie  rates,  minimized  aircraft  downtime,  and  maintenance  with 
minimum  support  resources.117 

Yet  the  1st  FW  identifies  this  system  as  undergoing  multiple 
software  versions  with  incompatibility  issues  with  different 
lots  of  aircraft.118  Perhaps,  similar  to  the  avionics  test-station 
failures  of  1976,  the  IMIS  (or  another  new  technology  on 
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the  F/A-22)  could  be  a  technology  choke  point  for  the 
F/A-22  in  2005. 

At  this  point,  a  direct  comparison  with  the  F-15A’s  IOC 
in  1976  versus  the  F/A-22  is  instructive.  Both  programs 
faced  issues  with  facilities,  sortie  production,  and  tech¬ 
nology.  With  both  aircraft  the  1st  FW  had  to  overcome  rela¬ 
tively  the  same  issues.  There  are,  however,  some  immediate 
differences. 

For  instance,  the  F/A-22’s  IOC  has  greater  access  to 
simulators  for  training  but  must  also  deal  with  aircraft 
modifications  at  Langley  AFB  that  produce  aircraft  down¬ 
time  and  affect  sortie  rates.  Another  area  of  difference  is 
the  meaning  of  IOC  declaration.  Langley  AFB  expects  to  re¬ 
ceive  its  17th  F/A-22  by  31  December  2005  and  declare 
IOC.119  In  contrast,  in  1976  Langley  AFB  received  its  24th 
F- 1 5A  and  1 7  days  later  deployed  to  Nellis  for  a  Red  Flag 
exercise.120  This  was  in  July  1976,  and  IOC  was  not  declared 
until  October,  with  one  squadron  IOC  and  another  IOC  by 
December.121  Clearly,  troops  in  1976  had  more  exposure  to 
the  F-15A  prior  to  IOC  than  the  men  and  women  at  Langley 
AFB  will  have  with  the  F/A-22  in  December  2005.  While 
that  may  seem  noteworthy,  the  key  is  the  definition  of  IOC. 
According  to  Headquarters  ACC,  “IOC  decision  is  based  on 
providing  an  operationally  credible  capability  to  the  com¬ 
batant  commanders  rather  than  providing  a  specified  num¬ 
ber  of  aircraft.”122 

In  short,  the  1st  TFW  in  1976  had  more  aircraft  for  more 
time  before  IOC  than  the  1st  FW  will  with  F /A-22s  in  2005. 
Did  this  translate  into  a  “better”  IOC  than  the  F /A-22  will 
have?  It  is  certainly  different,  but  the  greater  detail  of  the 
T&E  process  coupled  with  the  availability  of  the  ACS 
should  make  for  better  F /A-22  IOC  than  was  available  for 
the  F-15A.  Perhaps  the  cause  for  concern  lies  in  the  slips 
to  the  T&E  program  schedule  that  the  1st  FW  must  ulti¬ 
mately  accommodate  to  avoid  a  delay  of  IOC  declaration. 

Conclusions  and  Recommendations 

The  F/A-22’s  IOC  at  Langley  AFB  in  2005  is  a  momentous 
time  for  the  Combat  Air  Forces  (CAF).  It  represents  a  para¬ 
digm  shift  in  USAF  airpower  and,  like  the  F-15A’s  arrival 
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in  1976,  an  opportunity  for  change  that  may  not  be  re¬ 
peated  in  a  generation.  Lessons  from  the  F-15A’s  IOC  can 
be  used  as  a  comparison  and  measure  of  merit  against  the 
current  buildup  for  F/A-22  IOC.  If  developmental  or  IOC 
issues  that  occurred  with  the  F-15A  continue  with  the 
F/A-22,  then  observations  can  be  readily  made — either  the 
issues  are  inherent  with  a  new  aircraft’s  IOC  or  the  T&E 
system  has  not  adapted — or  perhaps  even  failed — to  learn 
from  past  lessons.  Langley’s  lessons  of  1976  can  either 
confirm  or  condemn  the  process  now  under  way  in  2005. 
Even  beyond  the  direct  comparison  between  the  two  air¬ 
craft,  the  Raptor’s  IOC  could  be  used  as  a  springboard  to 
further  needed  actions.  It  is  conceivable  that  this  comparison 
might  call  for  transformation  in  the  entire  T&E  process. 

Comparison  of  the  two  T&E  processes  reveals  that  both 
the  F-15A  and  the  F/A-22  used  the  same  model  for  T&E 
(DT  and  then  OT).  The  programs  faced  milestones  that 
were  used  to  keep  them  on  track  and  aligned  with  the  con¬ 
tracts.  Both  had  to  react  to  oversight  direction  (AIMVAL/ 
ACEVAL  and  F-15C  versus  F/A-22  CT),  needed  greater  op¬ 
erational  involvement  in  the  early  phases  of  the  testing 
(add-on  IOT&E  in  the  F- 1 5A  and  a  call  for  greater  MUT  in 
developmental  testing  for  the  F/A-22),  and  completed  their 
T&E  processes  right  before  IOC  at  Langley  AFB.  Testing  was 
more  extensive  and  technology  more  prominent  in  facili¬ 
tating  proficiency  at  IOC  for  the  F/A-22  than  for  the  F-15A. 
But  could  more  have  been  done? 

An  argument  could  be  made  that  “better”  IOC  is  needed — 
not  an  IOC  that  meets  a  rigid  timeline  but  one  that  truly 
represents  an  initial  combat  capability  to  the  COCOM.  Per¬ 
haps  IOC  declaration  needs  to  be  adjusted  to  the  delays 
that  the  T&E  process  (or  the  manufacturer)  causes.  Per¬ 
haps  IOC  should  be  a  more  flexible  target  date,  one  that 
moves  as  the  program  shifts.  Instead  of  asking  the  IOC 
base’s  leadership  to  “do  more  with  less,”  perhaps  the  IOC 
declaration  date  should  be  allowed  to  move.  Unfortunately, 
it  can  be  equally  well  argued  that  flexibility  in  the  IOC  dec¬ 
laration  date  is  unrealistic  because  it  is  the  key  milestone 
that  Congress  uses  to  ensure  that  a  program  it  funds  is 
implemented  as  it  directs  and  is  completed  on  time.  Ulti¬ 
mately,  the  point  to  debate  is  whether  or  not  a  hard  IOC 
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date  negatively  affects  organizational  behavior  during  de¬ 
velopment  or  at  the  IOC  base. 

Another  idea  is  to  consider  transforming  the  three- step 
T&E  process  to  update  the  model  that  the  F-15A  and 
F/A-22  used.  Surely  the  record  of  the  F/A-22  does  not  in¬ 
dicate  a  T&E  program  that  is  beyond  review.  In  2002  Mr. 
Thomas  P.  Christie,  the  director  for  OT&E  in  the  OSD  said, 
“I  took  part  in  the  DAB  review  in  the  fall  of  1986  that  ap¬ 
proved  its  [the  advanced  tactical  fighter]  entry  into  Demon¬ 
stration/Validation  (Dem/Val).  More  than  15  years  and  $27B 
later,  we’re  still  at  least  a  year  away  from  IOT&E  and  sev¬ 
eral  years  from  Full  Operational  Capability  for  the  F-22.”123 

Transformation,  defined  by  the  USAF  as  “a  process  by 
which  the  Air  Force  achieves  and  maintains  advantage 
through  changes  in  operational  concepts,  organizations, 
and/or  technologies  that  significantly  improve  its  war 
fighting  capabilities  or  ability  to  meet  the  demands  of  a 
changing  security  environment,”  is  the  watchword  for  the 
DOD  in  the  first  part  of  the  twenty-first  century.124  Could 
the  USAF’s  T&E  system  also  be  transformed?  Perhaps  the 
relationship  between  Air  Force  Materiel  Command  (AFMC) 
and  ACC  is  antiquated  and  needs  to  be  rethought.  The 
F/A-22’s  combined  DT/OT  to  achieve  economies  hints  at  a 
means  of  getting  more  out  of  T&E  data.  Could  not  the  call 
for  greater  MUT  in  DT  also  be  an  earmark  for  a  need  of  a 
new  model  or  construct  for  T&E?  Was  the  F/A-22’s 
OT&E(l)  a  fix  or  a  look  to  the  future? 

But  just  as  delaying  IOC  declaration  may  be  unrealistic, 
so  too  may  changes  to  the  T&E  process  be  impractical  be¬ 
cause  of  the  enormous  bureaucracy  and  needed  review 
that  is  associated  with  a  modern  weapons  system.  It  could 
be  that  acquisition  reform  or  transformation  is  not  possible 
because  of  Title  10  legal  requirements  mandated  by  Con¬ 
gress.  However  that  may  be,  findings  of  this  paper  indicate 
that  ideas  for  improvement  on  a  lesser  scale  could  be  made 
for  fighter  IOC  that  can  be  accomplished  at  the  MAJCOM 
level — ideas  that  are  more  practical  than  the  aforementioned 
grand  ones: 

1 .  Visit  the  last  base  to  declare  IOC.  Wings  that  are  ex¬ 
pecting  to  transition  to  a  new  weapons  system 
should  avail  themselves  of  the  experience  of  the  most 
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recent  base  to  go  through  the  same  process.  For  ex¬ 
ample,  the  1st  FW  should  observe  B-2  operations  at 
Whiteman  AFB,  Missouri,  and  learn  as  much  as  it 
can  about  the  B-2’s  IOC  process.  In  all  likelihood, 
certain  B-2  lessons  learned  will  be  applicable  to  the 
F/A-22  at  Langley  AFB. 

2.  Formally  hand  off  F/A-22  from  AFMC  to  ACC.  As  the 
T&E  process  reaches  milestones,  decisions  are  made 
based  on  available  information.  In  the  case  of  the 
F/A-22,  AFOTEC  produced  an  operational  assess¬ 
ment  for  LRIP  with  the  purpose  of  giving  decision 
makers  every  bit  of  information  and  a  recommenda¬ 
tion  for  the  LRIP  decision.  A  document  of  this  flavor 
could  be  created  and  declared  due  to  ACC  at  some 
time  before  IOC.  Certainly,  the  current  coordination 
between  the  involved  commands  and  agencies  is 
great.  But  no  document  could  be  located  that  said,  in 
effect,  “Langley,  these  are  the  problems,  issues,  and 
concerns  that  you  will  likely  have  with  the  F /A-22.  .  .  . 
The  developer  and  evaluator  recommend  the  following.” 
SATAFs,  PMRs,  T&E  reports,  and  transfer  of  trained 
personnel  are  all  great  steps — however,  a  formal  docu¬ 
ment/procedure  might  improve  the  process. 

3.  Form  an  IOC  advisory  board.  Numerous  organiza¬ 
tions  (e.g.,  the  US  Air  Force  Academy)  make  use  of  an 
advisory  board  that  has  the  ability  to  look  at  an  orga¬ 
nization  and  its  issues  with  the  perspective  of  expe¬ 
rience  and  distance.  Such  a  board  might  be  useful  in 
helping  the  IOC  wing  think  “outside  of  the  box”  and 
offer  positive  ideas.  Granted,  people  don't  want  anyone 
looking  over  their  shoulder,  neither  do  they  want  close 
oversight  of  the  “in  my  day  we  did  it  this  way”  variety 
by  senior  officers  or  extra  work  for  an  already  over¬ 
tasked  unit  trying  to  meet  a  deadline.  But  a  positive 
aspect  of  an  advisory-board  model  might  allow  issues 
to  be  rapidly  elevated  or  provide  senior  officers  with 
the  information  they  need  to  remain  comfortable  with 
a  project  with  political  oversight. 

4.  Conduct  F/A-22  road  shows.  Part  of  IOC  needs  to  be 
a  campaign  to  get  the  new  weapons  system  into  the 
CAF.  Other  weapons  systems  that  are  going  to  operate 
with  it  need  to  know  what  to  expect,  and  operational 
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and  strategic  planners  need  to  know  the  capabilities 
and  differences  of  the  new  aircraft.  The  1st  TFW  in 

1976  took  the  F-15A  on  the  road  to  Nellis  AFB  four 
times  before  IOC  in  order  to  affect  maintenance  and 
training.  A  secondary  effect  was  to  get  the  CAF  fa¬ 
miliar  with  the  new  aircraft.  The  same  could  be  true 
now  about  the  F/A-22.  It  represents  a  significant 
change  in  capability,  and  the  tactics  of  the  F/A-22 
will  likely  revolutionize  how  the  rest  of  the  CAF  fights 
with  it.  Certainly,  public  affairs  releases  can  “sell  the 
Raptor,”  but  getting  the  F/A-22  visible  in  the  field 
and  at  bases  across  the  United  States  will  expedite 
the  normalization  of  the  aircraft  in  the  USAF. 

5.  Capture  the  data  from  IOC.  Besides  histories,  the  single 
source  for  the  F-15A’s  IOC  at  Langley  AFB  was  written 
by  two  captains,  Lennon  and  Wray,  in  the  1st  TFW. 
Without  their  publication,  a  study  of  the  focused  IOC 
issues  would  be  extremely  difficult  and  research  in¬ 
tensive.  As  with  Lieutenant  Colonel  Lewis’s  History  of 
the  F-15  Eagle,  agencies  connected  with  the  F/A-22 
and  Langley  AFB  need  to  take  the  time  now  to  create 
a  document  that  records  the  lessons  learned  from 
development,  testing,  and  IOC  of  the  F/A-22.  It  is  in¬ 
teresting  to  note  that  Colonel  Lewis  wrote  his  history 
in  May  of  1976  (five  months  before  IOC),  and  Cap¬ 
tains  Wray  and  Lennon  wrote  their  study  in  June 

1977  (eight  months  after  IOC).  If  anything,  this  points 
to  the  need  to  record  the  events  while  the  memories 
are  still  recent. 

6.  Change  the  path  to  IOC.  While  changes  to  acquisition 
and  reform  for  T&E  may  be  unrealistic  due  to  the  en¬ 
trenched  nature  of  these  processes,  perhaps  changes 
to  IOC  and  its  definition  might  be  possible.  From  ACC’s 
perspective,  “ACC  has  developed  tiered  IOC  readi¬ 
ness  requirements  to  guide  and  monitor  progress  to¬ 
ward  IOC.  The  requirements  are  divided  into  four 
main  categories:  Aircraft  Status,  Operations,  Mainte¬ 
nance  &  Logistics,  and  Base  Support.”125  The  IOC 
definition  could  be  tightened  to  create  specific  meas¬ 
ures  of  effectiveness.  If  IOC  dates  cannot  be  changed 
once  determined,  then  a  balance  must  be  struck  be¬ 
tween  flexibility  and  combat  capability  provided  to 
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the  COCOM.  The  idea  behind  tighter  IOC  require¬ 
ments  might  serve  to  focus  energies  on  areas  that 
truly  matter — for  example,  a  checklist  of  achieve¬ 
ments  that  relates  to  the  ability  of  the  new  weapons 
system  to  perform  in  combat: 

a.  Aircraft  status:  number  of  jets  on  ramp,  comple¬ 
tion  of  OFP  changes,  or  hardware  modifications. 

b.  Operations:  training  production  from  the  FTU,  two 
completed  deployments  (Red  Flag  and  Weapons 
System  Evaluation  Program  [WSEP]). 

c.  Maintenance:  training  production  from  AETC,  UTE 
rate  goals  met,  maintenance  indicator  goals  met. 

d.  Base  support:  facilities  construction  complete,  first 
operational  readiness  inspection  (ORI)  complete. 

7.  Address  transformational  change.  Finally,  the  last  step 
of  the  IOC  process  might  be  to  step  back  from  the  de¬ 
tails  of  replacing  an  old  weapons  system  with  a  new 
one  and  determine  if  transformational  change  is 
needed.  If  the  F/A-22  brings  radical  change  in  war¬ 
fighting  capabilities  to  the  COCOM,  then  perhaps 
far-reaching  change  should  be  implemented.  For  ex¬ 
ample,  the  technology  of  the  F/A-22  may  need  orga¬ 
nizational  changes  in  the  1st  FW  because  of  new 
maintenance-repair  ideas.  Do  maintenance  squadrons 
need  to  change  functions,  names,  or  relationships? 
Do  the  operational  concepts  of  the  F/A-22  call  for  a 
change  to  the  F-15’s  fighter-  squadron  organization? 
Does  the  director  of  operations’  scheduling  shop  or 
weapons  shop  need  to  be  changed  or  augmented? 
Could  the  Global  Strike  Task  Force  concept  that  the 
F/A-22  is  part  of  call  for  a  different  wing  organiza¬ 
tion?  Maybe  air-refueling  tankers  need  to  be  part  of 
the  1st  FW  to  allow  the  F/A-22  to  execute  its  mis¬ 
sion.  Does  the  air-to-ground  mission  of  the  F/A-22 
demand  that  the  airspace  and  ranges  around  Langley 
AFB  be  changed  to  better  support  training  for  the 
F/A-22  mission?  What  about  adversaries  to  provide 
training  for  F/A-22  pilots?  If  the  F/A-22  is  “head- 
and-shoulders”  above  other  US  fighters,  then  the 
current  method  of  conducting  dissimilar  air  combat 
training  might  be  inadequate  for  the  F/A-22.  In 
short,  if  the  F/A-22  is  truly  transformational,  then 
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IOC  will  not  be  complete  without  out-of-the-box 
thinking  and  analysis  of  how  the  USAF  will  use  the 
F/A-22. 

In  summary,  the  approach  of  the  F/A-22’s  IOC  is  not  a 
time  to  lean  back  and  smile  like  the  parent  of  a  newborn 
child.  The  process  has  been  long,  yet  now  is  the  time  to 
capitalize,  to  push  out  the  old  ideas  of  the  F-15C,  and  to 
lay  the  foundation  for  rapid  growth  of  the  F/A-22  as  a 
weapons  system  that  will  mature  in  the  next  20  or  30  years. 
The  changes  can  be  radical — changing  the  T&E  model 
used  for  the  F-15A  and  the  F/A-22,  for  instance.  Or  they 
can  be  simple — perhaps  merely  capturing  the  history  of  the 
process  for  future  fighters  to  use.  Either  way,  IOC  needs  to 
go  beyond  “checking  the  squares”  and  determine  a  means 
of  pushing  for  the  transformation  that  the  F /A-22  provides 
the  CAF.  If  nothing  else,  these  recommendations  indicate 
that  there  are  still  more  questions  than  answers  for  CAF 
leaders  to  consider  as  they  plan  ahead  for  the  Raptors. 

Fighter  IOC  is  a  significant  event:  the  29-year  gap  between 
the  F-15A  and  the  F/A-22  indicates  that  USAF  leaders 
may  face  it  only  once  in  their  careers.  The  cost  and  invest¬ 
ment  alone  calls  for  singular  emphasis  and  insight.  Com¬ 
parison  of  the  F-15A’s  and  the  F/A-22’s  road  to  IOC  re¬ 
veals  fascinating  similarities  and  differences.  In  retrospect 
too  much  was  similar,  right  down  to  the  same  IOC  issues 
of  training,  sortie-rate  assumptions,  and  technology  choke 
points.  In  both  cases  the  wing  at  Langley  AFB  was  the  final 
step  in  the  process,  and  the  leadership — separated  by  29 
years — had  to  make  the  difference  in  order  for  IOC  to  be 
successful.  Surely  this  comparison  calls  for  the  USAF’s  vi¬ 
sion  of  transformation  to  be  applied  to  how  a  new  weapons 
system  reaches  IOC. 
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